MLLM-PINN-based wet friction element fault diagnosis method, system, medium and device

Abstract

This invention relates to the field of fault diagnosis of mechanical equipment components, and discloses a method, system, medium, and device for fault diagnosis of wet friction components based on MLLM-PINN. The method includes: acquiring three types of basic data (temperature, vibration, and text) of the wet friction component, preprocessing them, and inputting them into an MLLM multimodal feature fusion model to form a high-dimensional fusion feature containing visual, dynamic, and semantic information; constructing a PINN physical constraint model consisting of five fully connected hidden layers using PINN, inputting the high-dimensional fusion feature into the model to output fault category, physical parameter deviation, and damage degree; using a hybrid loss function to perform cross-model collaborative optimization of the MLLM multimodal feature fusion model and the PINN physical constraint model, and optimizing the model parameters to obtain the trained and optimized MLLM multimodal feature fusion model and PINN physical constraint model; and generating a structured diagnostic report for the wet friction component based on the physical prediction results of PINN and the semantic reasoning capability of a large language model.

Description

Technical Field

[0001] This invention relates to the field of mechanical equipment component fault diagnosis technology, and in particular to a method, system, medium and equipment for fault diagnosis of wet friction elements based on MLLM-PINN. Background Technology

[0002] As core transmission components in the transmission systems of mining equipment, heavy vehicles, and construction machinery, wet friction elements rely on the sliding friction between friction pairs for torque regulation and operating condition switching. Under conditions of high load and prolonged sliding friction, the friction interface of these components is prone to problems such as uneven temperature distribution, local buckling, abrasive wear, and warping induced by hot spots, directly impacting the system's operational safety and service life. Related research indicates that the thermo-mechanical coupling behavior of these components under complex operating conditions exhibits significant nonlinear and time-varying characteristics, necessitating more precise monitoring and diagnostic methods for health status assessment.

[0003] Existing research has analyzed the slippage process of wet clutches through experiments and numerical simulations. The results show that under conditions of insufficient local heat dissipation, a large radial temperature difference occurs at the friction interface, with a maximum difference exceeding 124.3℃. This temperature difference can induce hot spots and warping deformation in the mating steel plates. In the start-up phase study of large-size mining wet friction pairs, the contact pressure exhibits an S-shaped change, and the transient temperature distribution is significantly uneven in the radial direction. Although the highest temperature of the friction plates is lower than that of the mating steel plates, a significant temperature difference still exists between the oil groove and non-oil groove areas, which further expands after the system enters the unstable phase, becoming a significant factor leading to structural deformation. With the increase in equipment power density, traditional monitoring and assessment methods are no longer sufficient to meet the operational needs under complex working conditions. The technical demand for real-time monitoring, early warning, and source tracing of multi-dimensional damage characteristics is becoming increasingly prominent.

[0004] In the field of fault diagnosis of wet friction elements, numerous studies have been conducted both domestically and internationally, yielding certain results. Regarding temperature field prediction, the model incorporates the equivalent convective heat transfer coefficient and equivalent gain coefficient between the sub-units, comprehensively considering factors such as groove cooling, local heat dissipation, and friction coefficient variations, reducing the maximum deviation between simulation results and experimental values ​​to 5.7℃. Existing technologies further indicate that the temperature field is influenced by factors such as interfacial pressure, relative rotational speed, and convective heat transfer, with a significant radial temperature gradient and a peak typically located in the outer diameter region. In terms of numerical solution of the physical field, a hybrid PINN (Physical Information Neural Network) model combining strong and weak forms is used to solve heterogeneous thermo-mechanical coupling problems, improving the prediction accuracy of temperature and stress fields. The above research demonstrates that PINN can solve both the forward and inverse problems of the friction contact temperature field using a small amount of measured data, and can deduce key parameters such as the heat distribution coefficient (HPC) and convective heat transfer coefficient (CHTC), reducing the dependence of traditional models on boundary conditions.

[0005] In damage modeling and prediction, domestic researchers have constructed various models based on data fusion and intelligent algorithms. Using a PSO-BP neural network with rotational speed and engagement oil pressure as inputs, they have achieved predictions of multiple damage characteristics, including the circumferential temperature gradient of the friction plate, the rate of change of Fe / Cu elemental concentration, and the rate of change of surface roughness, achieving a model fit of 96.7%. For small-sample fault data, a combination of PINN and diffusion networks has been proposed, extracting time-frequency features through wavelet transform and enhancing the data using diffusion networks, significantly improving fault diagnosis performance. International research has designed a transferable friction model based on PINN. Under the physical constraints of the LuGre model, it can accurately obtain friction characteristics using only a small amount of noisy data and can be extended to untrained systems. An improved Hilbert-Huang transform (IHHT), combined with wavelet packet transform and empirical mode decomposition, enhances the instantaneous feature extraction capability, achieving a buckling damage recognition rate of 94.7%.

[0006] In terms of intelligent diagnostic methods, the application of Multimodal Large Language Model (MLLM) and PINN provides a new technical path for fault identification under complex working conditions. Among existing technologies, the DiagLLM framework, based on the fusion of visual and textual information and employing LoRA for efficient fine-tuning, achieves a diagnostic accuracy of 98.44% in small-sample bearing fault scenarios, providing a reference for multimodal diagnosis of wet friction components. Combining GAN-generated samples with inverse PINN physical constraint inference effectively solves the problems of data imbalance and rare fault identification. The multi-sensor raw signal fusion method utilizes CNN to automatically extract features, achieving a diagnostic accuracy exceeding 99% in bearing and gearbox experiments. Furthermore, research on PINN coupling training strategies demonstrates that hybrid PINN achieves better accuracy than traditional PINN and DEM methods in solving heterogeneous thermo-mechanical coupling problems, providing a technical foundation for multi-physics field diagnosis of friction components.

[0007] Despite the progress made in existing technologies, there are still several shortcomings in the intelligent diagnosis of wet friction elements: (1) The accuracy of physical field modeling is limited, especially the dynamic characteristics of contact pressure and the coupling effect of thermal load and wear are not fully reflected, resulting in deviations in thermal failure judgment and damage threshold prediction; (2) The utilization efficiency of multi-source data is not high, most methods rely on a single mode, and it is difficult to cope with the problems of sample scarcity and data imbalance; (3) Deep models have limited performance in terms of interpretability and cross-condition generalization, and PINN has training instability and parameter optimization difficulties in complex coupling problems; (4) The diagnostic system has not formed a complete technology chain and lacks an integrated solution from condition monitoring to maintenance decision-making. Summary of the Invention

[0008] To address the aforementioned issues, the present invention aims to provide a method, system, medium, and device for fault diagnosis of wet friction elements based on MLLM-PINN. This method effectively addresses engineering challenges such as small sample diagnosis under complex operating conditions, insufficient physical consistency constraints, and fragmented operation and maintenance processes, significantly improving the reliability and engineering applicability of fault diagnosis for wet friction elements.

[0009] To achieve the above objectives, in a first aspect, the technical solution adopted by the present invention is as follows: a fault diagnosis method for wet friction elements based on MLLM-PINN, comprising: acquiring three types of basic data of the wet friction element—temperature, vibration, and text; preprocessing the three types of basic data; inputting them into an MLLM multimodal feature fusion model; obtaining implicit correlations through a visual and semantic alignment mechanism; forming a high-dimensional fusion feature containing visual, dynamic, and semantic information to enhance the completeness of fault feature expression; constructing a PINN physical constraint model consisting of five fully connected hidden layers using PINN; inputting the high-dimensional fusion feature into the PINN physical constraint model; and outputting the fault category and physical parameter deviations. The differences and damage levels are assessed to ensure reliable constraints on the physical mechanisms during the diagnostic process. A hybrid loss function, consisting of data loss, physical loss, and cross-modal fusion loss, is used to perform cross-model collaborative optimization of the MLLM multimodal feature fusion model and the PINN physical constraint model. The Adam optimizer is then used to optimize the model parameters, and an iterative learning rate annealing strategy is employed to obtain the trained and optimized MLLM multimodal feature fusion model and PINN physical constraint model. Based on the trained MLLM multimodal feature fusion model and PINN physical constraint model, the physical prediction results of PINN and the semantic reasoning ability of the large language model are obtained to generate a structured diagnostic report for wet friction elements.

[0010] Furthermore, three types of basic data—temperature, vibration, and text—are acquired from the wet friction element, and preprocessing is performed on these three types of basic data, including: After obtaining temperature information through silicon carbide crystal thermometry, the Unet network is used to reconstruct the field of local point data to form a complete temperature field image. Vibration signals are acquired by sensors, and time-frequency feature maps are extracted through wavelet packet transform; The text data consists of operation and maintenance records, fault manuals, and expert experience, and is encoded into vector features using the BERT model.

[0011] Furthermore, the input to the MLLM multimodal feature fusion model is used to obtain implicit relationships through visual and semantic alignment mechanisms, forming high-dimensional fused features that include visual, dynamic, and semantic information, including: A multimodal large language model structure is adopted, based on DiagLLM, and combined with LoRA low-rank adaptation technology to achieve parameter fine-tuning; In the fusion stage, the visual encoder of DiagLLM receives the time-frequency feature map obtained by converting the temperature field image and vibration signal, the text encoder processes the vector features, and the alignment and association of multimodal features are realized through the cross-attention mechanism to generate a high-dimensional fused feature vector. Consistency verification is performed by calculating the cosine similarity between the fused features and the single-modal features, and the similarity of each feature is required to be no less than the set threshold.

[0012] Furthermore, a PINN physical constraint model consisting of five fully connected hidden layers is constructed using PINN, including: Input layer; The first hidden layer consists of 128-dimensional fully connected units, using the Swish activation function and configured with a Dropout regularization with a dropout rate of 0.2; The second hidden layer consists of 256-dimensional fully connected units, using the same activation function and regularization settings; The third hidden layer is a 512-dimensional fully connected unit, which continues to use Swish activation and Dropout with a dropout rate of 0.2; The fourth hidden layer consists of 256-dimensional fully connected units, maintaining the same activation and regularization strategies; The fifth hidden layer consists of 128-dimensional fully connected units, with consistent activation and regularization settings; The network ends with a 6-dimensional output layer, used to characterize the physical quantity to be determined; The network incorporates three types of physical constraints: the heat conduction equation of wet friction elements, the S-shaped variation law of contact pressure, and the friction torque transmission model.

[0013] Furthermore, a hybrid loss function consisting of data loss, physical loss, and cross-modal fusion loss is adopted:

[0014] In the formula, Total loss; This results in a loss of data consistency. For fusion loss; This represents the loss due to physical constraints.

[0015] Furthermore, a hybrid loss function consisting of data loss, physical loss, and cross-modal fusion loss is employed to perform cross-model co-optimization of the MLLM multimodal feature fusion model and the PINN physical constraint model, comprising three stages: During training epochs 1–50, the main parameters of PINN are fixed, and only the LoRA adaptation layer of MLLM is updated to minimize the fusion loss. Loss of data consistency ; During training epochs 51–150, the MLLM parameters are kept constant, and only the gradients of PINN are updated to optimize the physical constraint loss. and data consistency loss ; In training epochs 151–200, parameter freezing was removed, and MLLM and PINN were trained simultaneously to optimize the total loss function. ; During training, the model checkpoint is saved every set number of rounds, and the model with the minimum total loss in the validation set is used as the final output after all training is completed.

[0016] Furthermore, based on the trained MLLM multimodal feature fusion model and PINN physical constraint model, the physical prediction results of PINN and the semantic reasoning ability of the large language model are obtained to generate a structured diagnostic report of the wet friction element, including: The acquired temperature data, vibration signals and operating condition text are preprocessed to generate temperature field images, vibration time-frequency diagrams and corresponding text vectors. Cross-modal representations are constructed using MLLM, and fused features are output. PINN then infers fault-related physical parameters based on these features. MLLM is then matched with a structured knowledge base to establish a mapping relationship between physical parameter deviations and potential fault causes, and finally a structured diagnostic report is generated and sent back to the operation and maintenance terminal. The diagnostic report includes the diagnosis time, operating parameters, fault type and confidence level, degree of damage, deviation of key physical parameters, fault cause analysis, and short-term and long-term maintenance recommendations and re-inspection cycle.

[0017] Secondly, the technical solution adopted by this invention is as follows: a fault diagnosis method for wet friction elements based on MLLM-PINN, comprising: a data preprocessing and cross-modal feature fusion module, which acquires three basic data types of wet friction elements: temperature, vibration, and text, and after preprocessing the three basic data types, inputs them into the MLLM multimodal feature fusion model, obtains implicit correlations through a visual and semantic alignment mechanism, and forms high-dimensional fusion features containing visual, dynamic, and semantic information to enhance the completeness of fault feature expression; and a physical constraint modeling module, which uses PINN to construct a PINN physical constraint model consisting of five fully connected hidden layers, and inputs the high-dimensional fusion features into the PINN physical constraint model, outputting fault category and physical parameter deviations. The system assesses the difference and degree of damage to ensure reliable constraints on the physical mechanism during the diagnostic process. A cross-model collaborative optimization module employs a hybrid loss function consisting of data loss, physical loss, and cross-modal fusion loss to collaboratively optimize the MLLM multimodal feature fusion model and the PINN physical constraint model. The Adam optimizer is used to optimize the model parameters, and an iterative learning rate annealing strategy is employed to obtain the trained and optimized MLLM multimodal feature fusion model and PINN physical constraint model. The diagnostic output module, based on the trained MLLM multimodal feature fusion model and PINN physical constraint model, obtains the physical prediction results of PINN and the semantic reasoning ability of the large language model, generating a structured diagnostic report for the wet friction element.

[0018] Thirdly, the technical solution adopted by the present invention is: a computer-readable storage medium for storing one or more programs, wherein the one or more programs include instructions, which, when executed by a computing device, cause the computing device to perform any of the methods described above.

[0019] Fourthly, the technical solution adopted by the present invention is: a computing device comprising: one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include instructions for performing any of the methods described above.

[0020] The present invention has the following advantages due to the adoption of the above technical solutions: 1. This invention is based on the dual-drive fault diagnosis framework of MLLM-PINN, which overcomes the bottleneck of insufficient semantic understanding or limited physical interpretability of traditional single models by cross-modal fusion and physical constraint modeling.

[0021] 2. This invention adopts a three-modal fusion mechanism of temperature field, vibration, and operation and maintenance text, and uses prior information from the text knowledge base to make up for the defects of small sample data and enhance the integrity of fault features.

[0022] 3. This invention constructs a triple loss optimization strategy of "data-physics-fusion" and embeds physical equations in PINN to ensure that the diagnostic results are both consistent with the measured data and in accordance with engineering principles.

[0023] 4. This invention establishes a closed-loop diagnostic system that enables integrated output from feature extraction to maintenance decisions, automatically generates structured reports, and lowers the threshold for operation and maintenance. Attached Figure Description

[0024] Figure 1 This is a flowchart of the wet friction element fault diagnosis method based on MLLM-PINN in an embodiment of the present invention; Figure 2 This is a schematic diagram of the PINN fault diagnosis network topology and output dimension based on MLLM fusion features in an embodiment of the present invention. Detailed Implementation

[0025] To address the current problems in wet friction element fault diagnosis, such as insufficient accuracy of physical field modeling, low fusion of multi-source monitoring information, and weak interpretability of diagnostic models, this invention proposes a method, system, medium, and equipment for wet friction element fault diagnosis based on MLLM-PINN. This invention constructs a multimodal data fusion mechanism, introduces physical constraints reflecting the friction transmission law, and employs a collaborative optimization strategy to achieve information interaction and result correction between models, thereby realizing accurate identification of wet friction element fault states, cause analysis, and generation of operation and maintenance strategies. This invention effectively solves the aforementioned technical problems by combining MLLM and PINN. For example, DiagLLM provides a unified fusion framework applicable to multi-source monitoring data such as temperature, vibration, and oil; PINN can improve model stability and boundary condition inference capabilities through physical constraints; and the GAN-inverse PINN model can enhance adaptability to small sample sizes and rare fault scenarios. Based on domestic research in multi-source feature extraction and data fusion, it is necessary to construct an integrated diagnostic method that integrates multimodal data, possesses physical constraint capabilities, and can cover the entire process of operation and maintenance tasks, in order to make up for the shortcomings of the existing technical system and improve the operational reliability of wet friction elements under complex working conditions.

[0026] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the described embodiments of the present invention are within the scope of protection of the present invention.

[0027] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments according to this application. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0028] In one embodiment of the present invention, a method for fault diagnosis of wet friction elements based on MLLM-PINN is provided. This method uses a loss function feedback to fuse a multimodal large language model (MLLM) and a physical information neural network (PINN) to process multi-source data (image, vibration, and text) for fault diagnosis of wet friction elements. In this embodiment, a closed-loop technical system is constructed, consisting of "multi-source data acquisition and input—MLLM multimodal feature fusion—PINN physical constraint modeling—hybrid optimization training—diagnosis and decision output," and the functional positioning, data flow, and coupling relationships of each module are clearly defined. Specifically, as follows... Figure 1 As shown, the method includes the following steps: 1) Acquire three basic data types of wet friction elements: temperature, vibration, and text. After preprocessing the three basic data types, input them into the MLLM multimodal feature fusion model. Obtain implicit correlations through visual and semantic alignment mechanisms to form high-dimensional fusion features containing visual, dynamic, and semantic information, thereby enhancing the completeness of fault feature expression. 2) A PINN physical constraint model consisting of five fully connected hidden layers is constructed using PINN. High-dimensional fusion features are input into the PINN physical constraint model to output fault category, physical parameter deviation and damage degree, so as to realize reliable constraint of physical mechanism in the diagnostic process; 3) A hybrid loss function consisting of data loss, physical loss, and cross-modal fusion loss is used to perform cross-model collaborative optimization of the MLLM multimodal feature fusion model and the PINN physical constraint model. The Adam optimizer is used to optimize the model parameters, and the learning rate annealing strategy is combined for iteration until the validation set loss tends to converge in several consecutive rounds and the decrease is less than the set threshold, so as to ensure the stable convergence of the model and obtain the trained and optimized MLLM multimodal feature fusion model and PINN physical constraint model. 4) Based on the trained MLLM multimodal feature fusion model and PINN physical constraint model, obtain the physical prediction results of PINN and the semantic reasoning ability of the large language model to generate a structured diagnostic report of wet friction element.

[0029] In step 1) above, three types of basic data—temperature, vibration, and text—of the wet friction element are obtained, and these three types of basic data are preprocessed. The specific implementation process is as follows: (1) After the temperature information is obtained by the silicon carbide crystal thermometry method, the Unet network is used to reconstruct the field of local point data to form a complete temperature field image.

[0030] In this embodiment, the sensors are arranged using a combination of circumferential uniform distribution and radial layering. Five independent temperature measurement channels are set along the circumference of the friction plate, with each channel maintaining basic spatial symmetry. In the radial direction, based on the structural characteristics of the friction plate, measurement points are arranged in three regions: the outer diameter, the sub-outer diameter, and the inner diameter, with a spacing of no less than 15 mm between each point to ensure the representativeness of the temperature field distribution measurement. The high-temperature resistant fiber optic grating sensor used for temperature measurement is fixed in the pre-set mounting holes of the friction plate via a ceramic base. The distance between the sensor probe and the friction interface is controlled within 2 mm to reduce the influence of the interface temperature gradient on the measurement results. The fiber optic signal is led out through a dedicated rotary connector, a structure that avoids fiber optic pulling or entanglement damage caused by the high-speed rotation of the friction plate. The transmission link uses single-mode fiber to meet the signal transmission distance requirement of no more than 100 m; all signals are ultimately connected to an FBG demodulator for real-time acquisition and processing of temperature data.

[0031] Data Acquisition Process: Test conditions were set according to the typical operating range of a mining wet clutch, with the speed difference controlled between 200 and 900 r / min, engagement oil pressure between 0.2 and 0.7 MPa, and a single slippage duration of approximately 30 s. Under these conditions, the five temperature measurement points were simultaneously acquired using an FBG demodulator, with a sampling frequency set to 100 Hz, covering the entire slippage process. All measurement results were recorded in chronological order and stored as a CSV file, containing timestamps and temperature data for each measurement channel. After acquisition, the temperature sequence was preprocessed. First, a moving average method with a window length of 5 was used to suppress high-frequency noise; then, a temperature compensation algorithm was introduced to correct deviations caused by ambient temperature drift, thereby improving the effectiveness and consistency of the temperature field data. The compensation formula is as follows:

[0032] in, Ambient temperature; Indicates the surface temperature of a wet friction element; This represents the actual temperature after ambient temperature compensation.

[0033] Temperature field reconstruction stage: A three-dimensional thermal analysis model of the wet friction plate was established based on ANSYS Workbench. Boundary conditions such as frictional heat generation power, material thermal conductivity coefficient, and oil convective heat transfer coefficient were set in the model. Numerical simulations were conducted under different fault conditions (including hot spots, localized heat dissipation limitation, and abnormal pressure), generating approximately 1500 sets of temperature field distribution data with reference accuracy, which were used as training samples for the Unet neural network. The Unet network adopts an encoder-decoder structure, and its training parameters are set as follows: learning rate 5×10⁻⁶. -4 The batch size was 8, the training epochs were 150, and the loss function was mean squared error. The training process used a convergence criterion that the temperature prediction error on the validation set did not exceed 3%. After model training, the preprocessed five-channel measured temperature sequence was input into the Unet network to reconstruct a 200×200 pixel temperature field distribution map. The resulting image was normalized and saved in PNG format as one of the input data for the multimodal large language model.

[0034] (2) The vibration signal is collected by the sensor and the time-frequency feature map is extracted by wavelet packet transform.

[0035] In this embodiment, three acceleration sensors are evenly arranged circumferentially around the outer periphery of the wet clutch housing, with an included angle of 120° between them. The mounting surface of each sensor is parallel to the end face of the friction plate, and the axial distance between the sensor and the end face of the friction plate is no greater than 50 mm. The sensors are installed using M5 threads with a mounting torque of 2 N·m to ensure installation rigidity and measurement stability. Vibration signals are synchronously acquired via a data acquisition card at a sampling frequency of 5 kHz, with a single acquisition duration of 30 s. The acquisition is triggered by a slip-initiation signal. The acquired raw data is stored in CSV format.

[0036] In the preprocessing stage, a 3×3 windowed mid-range filter is first used to remove impulse noise, followed by a db4 wavelet basis (decomposition level of 5, threshold). ,in The standard deviation of noise. Wavelet thresholding noise reduction was performed on the data (to a specified length), followed by wavelet packet transform decomposition into 8 layers to obtain 128 frequency bands. The energy entropy H of each frequency band was calculated.

[0037] in, Let i be the energy percentage of the i-th frequency band, and calculate the peak factor C:

[0038] in, This is the maximum amplitude value within the frequency band. The root mean square amplitude is used as the final value. Finally, a 128×128 pixel time-frequency feature map is generated, normalized, and saved as a PNG format.

[0039] (3) The text data consists of operation and maintenance records, fault manuals and expert experience, and is encoded into vector features using the BERT model.

[0040] In this embodiment, 5,000 operation and maintenance records of mining wet friction elements were collected, covering dimensions such as basic equipment information, operating condition parameters, fault correlation information, quantitative data of damage characteristics, maintenance and repair records, lubrication system parameters, and environmental and monitoring data. Official technical manuals for three types of core faults (hot spot warping, buckling deformation, and abrasive wear) were also compiled, along with 100 knowledge mapping experiences on "fault phenomena—physical parameter deviations—handling solutions" provided by 10 senior engineers. Structured key fields were extracted from the above text using regular expressions to construct a knowledge base containing information such as fault type, speed difference, oil pressure, temperature gradient, contact pressure deviation, and corresponding handling solutions.

[0041] The text representation employs the BERT-base-uncased model. The text is segmented and encoded using a tokenizer, and the mean of the last hidden state is taken as the text semantic vector, resulting in a 768-dimensional feature representation. The generated text vectors and their corresponding labels are uniformly written to a MySQL database for subsequent MLLM fine-tuning training.

[0042] All of the above-mentioned data undergo normalization, noise reduction, and format standardization before entering subsequent stages to ensure the stability of the overall modeling.

[0043] In step 1) above, the preprocessed data is input into the MLLM multimodal feature fusion model. Implicit relationships are obtained through visual and semantic alignment mechanisms, forming a high-dimensional fusion feature containing visual, dynamic, and semantic information. This includes the following steps: 1.1) A multimodal large language model structure is adopted, based on DiagLLM, and combined with LoRA low-rank adaptation technology to achieve parameter fine-tuning; 1.2) In the fusion stage, the visual encoder of DiagLLM receives the time-frequency feature map obtained by converting the temperature field image and vibration signal, the text encoder processes the vector features, and the alignment and association of multimodal features are realized through the cross-attention mechanism to generate a high-dimensional fusion feature vector. 1.3) Consistency verification is performed by calculating the cosine similarity between the fused features and the single-modal features. The similarity of each feature must not be lower than the set threshold. Specifically, the similarity of temperature field features must not be lower than 0.82, the similarity of vibration features must not be lower than 0.80, and the similarity of text features must not be lower than 0.78.

[0044] In this embodiment, a multimodal pairing dataset of 8000 sets is constructed. The data includes temperature field images, vibration time-frequency graphs, text vectors, fault type labels, and physical parameter labels. The data is divided into training, validation, and test sets according to a conventional ratio. The image modality is enhanced by horizontal or vertical flipping, 90° / 180° / 270° rotation, and Gaussian noise with a variance of 0.01. The text modality expands the sample size through synonym replacement and word order adjustment, increasing the overall data volume to 10000 sets.

[0045] For model training, DiagLLM with a parameter size of 7B was used as the basic framework, and LoRA low-rank adaptation was employed for efficient fine-tuning. The main model parameters were kept frozen, and only the low-rank matrix with a rank of 8 was updated, with the adaptation occurring at the Transformer's attention mechanism layer. The training batch size was set to 16, the training epochs to 50, and the optimizer to AdamW. =0.9, =0.999), with an initial learning rate of 5×10. -5 And adjust to a minimum of 5×10 using cosine annealing. -6 Training was performed on a computer using PyTorch, taking approximately 24 hours. The training process used a cross-modal alignment similarity of at least 0.85 on the validation set as the convergence criterion.

[0046] In the fusion stage, the DiagLLM visual encoder receives temperature field and time-frequency images, while the text encoder processes 768-dimensional vector features. A cross-attention mechanism is used to align and associate multimodal features, generating a 2048-dimensional fused feature vector. Consistency is verified by calculating the cosine similarity between the fused features and the individual modal features. The similarity requirement is no less than 0.82 for temperature field features, no less than 0.80 for vibration features, and no less than 0.78 for text features.

[0047] In step 2) above, a Physical Information Neural Network (PINN) is used to construct a network structure consisting of five fully connected hidden layers, and the Swish activation function is used to enhance nonlinear modeling capabilities. Specifically, a PINN physical constraint model consisting of five fully connected hidden layers is constructed using PINN, and the PINN network model is built based on TensorFlow 2.15, as follows: Figure 2 As shown, the network topology consists of the following structures from the input end to the output end: The input layer is set to 2048 dimensions; The first hidden layer consists of 128-dimensional fully connected units, using the Swish activation function and configured with a Dropout regularization with a dropout rate of 0.2; The second hidden layer consists of 256-dimensional fully connected units, using the same activation function and regularization settings; The third hidden layer is a 512-dimensional fully connected unit, which continues to use Swish activation and Dropout with a dropout rate of 0.2; The fourth hidden layer consists of 256-dimensional fully connected units, maintaining the same activation and regularization strategies; The fifth hidden layer consists of 128-dimensional fully connected units, with consistent activation and regularization settings; The network ends with a 6-dimensional output layer, used to characterize the physical quantity to be determined; The network incorporates three types of physical constraints: the heat conduction equation of wet friction elements, the S-shaped variation law of contact pressure, and the friction torque transmission model.

[0048] In this embodiment, the three types of physical constraints are as follows: First, the transient heat conduction equation:

[0049] In the formula, =7850kg / m³, c=460J / (kg·K), K=50W / (m·K), Let represent the material density of the wet friction element, c represent the specific heat capacity of the wet friction element material, and K represent the thermal conductivity of the wet friction element material. This is represented as the temperature field of a wet friction element. This is represented by the temperature gradient of the temperature field in a wet friction element. This represents the frictional heat flux. The partial derivatives are then calculated using TensorFlow's automatic differentiation function, along with the physical losses. for:

[0050] The number of physical constraint sampling points is N=1000; This represents the temperature field of the wet friction element at the i-th physical constraint sampling point; It represents the frictional heat flux at the i-th physical constraint sampling point.

[0051] Second, the equation for the S-shaped curve of contact pressure:

[0052] In the formula, p0 = 0.5 MPa, k = 0.1, t0 = 10 s, p0 represents the saturation value of the contact pressure of the wet friction element, k represents the rate of change coefficient of the S-curve of the contact pressure, and t0 represents the inflection point time of the S-curve of the contact pressure. And physical losses... for:

[0053] The number of time sampling points is M=300; This represents the theoretical value of the contact pressure calculated from the S-curve equation of the contact pressure at the j-th time sampling point. It represents the contact pressure value predicted by the model at the j-th time sampling point.

[0054] Thirdly, the formula for transmitting frictional torque:

[0055] and For contact area, For the effective radius, This represents the coefficient of friction for wet friction elements. This is expressed as the contact pressure of a wet friction element. This represents the frictional torque transmitted by a wet friction element, and includes physical losses. for:

[0056] The number of torque sampling points is K=200. This represents the sampled value of the contact pressure of the friction element at the m-th torque sampling point. Let the measured friction torque at the m-th torque sampling point be represented as the total physical loss. for:

[0057] During the training phase, the fused features output by the MLLM are used as input and paired with the corresponding labels, with a learning rate of 1×10⁻⁶. -4 The batch size is 32, the training rounds are 200, an early stopping strategy is adopted, the loss on the test and validation sets does not decrease for 10 consecutive rounds, and the trained model is saved in .h5 format.

[0058] In step 3) above, the hybrid loss function, consisting of data loss, physical loss, and cross-modal fusion loss, is as follows:

[0059] In the formula, Total loss; This is the data consistency loss, used to assess the deviation between predicted values ​​and true labels; The fusion loss is used to balance the feature correspondence between the multimodal model and PINN; This is the physical constraint loss, used to ensure that the model output meets the corresponding physical constraints.

[0060] Among them, data consistency loss for:

[0061] in, This represents the model's predicted diagnostic parameter value for the nth output dimension; 6 represents the true diagnostic parameter label value of the nth output dimension; 6 represents the output dimensions: fault type (1 dimension), damage degree (1 dimension), heat distribution coefficient (1 dimension), convective heat transfer coefficient (1 dimension), temperature gradient (1 dimension), and contact pressure deviation (1 dimension).

[0062] Fusion loss for:

[0063] in, The fusion features of MLLM output, This is the input feature for PINN.

[0064] In this embodiment, a hybrid loss function consisting of data loss, physical loss, and cross-modal fusion loss is used to perform cross-model collaborative optimization of the MLLM multimodal feature fusion model and the PINN physical constraint model, including three stages: Phase 1: During training epochs 1–50, the main parameters of PINN are fixed, and only the LoRA adaptation layer of MLLM is updated to minimize the fusion loss. Loss of data consistency .

[0065] Phase 2: During training epochs 51–150, the MLLM parameters are kept constant, and only the gradients of PINN are updated to optimize the physical constraint loss. and data consistency loss .

[0066] Phase 3: During training epochs 151–200, parameter freezing is lifted, and MLLM and PINN are trained simultaneously to optimize the total loss function. .

[0067] During training, a model checkpoint is saved every set number of rounds (e.g., 20 rounds), and the model with the minimum total loss in the validation set is used as the final output after all training is completed.

[0068] In step 4) above, in this embodiment, the hardware is deployed on an edge computing node, running the Ubuntu 22.04 operating system, the diagnostic model is encapsulated through Docker containers, and an API service interface is built based on the Flask framework; real-time data transmission adopts the MQTT protocol to meet the requirements of low latency and high reliability in the field environment.

[0069] Specifically, based on the trained MLLM multimodal feature fusion model and PINN physical constraint model, the physical prediction results of PINN and the semantic reasoning ability of the large language model are obtained to generate a structured diagnostic report for wet friction elements, including the following steps: 4.1) The acquired temperature data, vibration signals and operating condition text are preprocessed to generate temperature field images, vibration time-frequency diagrams and corresponding text vectors.

[0070] 4.2) Cross-modal representation is constructed using MLLM, and fused features are output; then PINN infers fault-related physical parameters based on these features.

[0071] 4.3) MLLM is then matched with the structured knowledge base to establish a mapping relationship between physical parameter deviations and potential fault causes, and finally a structured diagnostic report is generated and sent back to the operation and maintenance terminal.

[0072] In this embodiment, MLLM is then matched with a structured knowledge base to establish a mapping relationship between physical parameter deviations and potential fault causes. Specifically, MLLM performs deep semantic matching with a built-in structured fault knowledge base for wet friction elements. This knowledge base pre-stores association rules for "physical parameter deviation combinations - potential fault causes" extracted from massive operation and maintenance records. MLLM encodes the 6-dimensional physical parameter deviation tensor output by PINN into a semantic feature vector, performs cosine similarity calculation and matching degree ranking with the rule entries in the knowledge base, filters high-confidence association entries, and thus establishes a precise mapping relationship between multi-dimensional physical parameter deviations and single / compound fault causes.

[0073] The diagnostic report includes the diagnosis time, operating condition parameters, fault type and confidence level, damage degree, deviation of key physical parameters, fault cause analysis, and short-term and long-term maintenance recommendations and re-inspection cycle, thus forming a complete closed-loop support system from data collection and feature extraction to operation and maintenance decision-making.

[0074] For example, when the fault type is identified as hot spot warping (confidence level of 98.7%), the damage level is 65 points, and the temperature gradient is 35 ℃ / mm, the system determines that the cause is insufficient local heat dissipation capacity of the contact surface; accordingly, it provides short-term suggestions (such as adjusting the lubrication flow rate to 15 L / min), long-term suggestions (such as adopting a fan-shaped oil groove structure with enhanced heat dissipation capacity), and a re-inspection cycle (50 working cycles).

[0075] In one embodiment of the present invention, a method for fault diagnosis of wet friction elements based on MLLM-PINN is provided, comprising: The data preprocessing and cross-modal feature fusion module acquires three basic data types of wet friction elements: temperature, vibration, and text. After preprocessing these three types of basic data, the data is input into the MLLM multimodal feature fusion model. Implicit correlations are obtained through a visual and semantic alignment mechanism, forming a high-dimensional fusion feature that includes visual, dynamic, and semantic information to enhance the completeness of fault feature expression. The physical constraint modeling module uses PINN to construct a PINN physical constraint model consisting of five fully connected hidden layers. High-dimensional fusion features are input into this PINN physical constraint model, and the fault category, physical parameter deviation, and damage degree are output to achieve reliable constraints on the physical mechanism during the diagnostic process. The cross-model collaborative optimization module uses a hybrid loss function consisting of data loss, physical loss, and cross-modal fusion loss to perform cross-model collaborative optimization of the MLLM multimodal feature fusion model and the PINN physical constraint model. It also uses the Adam optimizer to optimize the model parameters and combines the learning rate annealing strategy for iterative optimization to obtain the trained and optimized MLLM multimodal feature fusion model and PINN physical constraint model. The diagnostic output module, based on the trained MLLM multimodal feature fusion model and PINN physical constraint model, obtains the physical prediction results of PINN and the semantic reasoning ability of the large language model, and generates a structured diagnostic report of wet friction elements.

[0076] In the above embodiments, three types of basic data—temperature, vibration, and text—of the wet friction element are acquired, and these three types of basic data are preprocessed, including: After obtaining temperature information through silicon carbide crystal thermometry, the Unet network is used to reconstruct the field of local point data to form a complete temperature field image. Vibration signals are acquired by sensors, and time-frequency feature maps are extracted through wavelet packet transform; The text data consists of operation and maintenance records, fault manuals, and expert experience, and is encoded into vector features using the BERT model.

[0077] In the above embodiments, the input MLLM multimodal feature fusion model obtains implicit relationships through visual and semantic alignment mechanisms, forming high-dimensional fusion features containing visual, dynamic, and semantic information, including: A multimodal large language model structure is adopted, based on DiagLLM, and combined with LoRA low-rank adaptation technology to achieve parameter fine-tuning; In the fusion stage, the visual encoder of DiagLLM receives the time-frequency feature map obtained by converting the temperature field image and vibration signal, the text encoder processes the vector features, and the alignment and association of multimodal features are realized through the cross-attention mechanism to generate a high-dimensional fused feature vector. Consistency verification is performed by calculating the cosine similarity between the fused features and the single-modal features, and the similarity of each feature is required to be no less than the set threshold.

[0078] In the above embodiments, a PINN physical constraint model consisting of five fully connected hidden layers is constructed using PINN, including: Input layer; The first hidden layer consists of 128-dimensional fully connected units, using the Swish activation function and configured with a Dropout regularization with a dropout rate of 0.2; The second hidden layer consists of 256-dimensional fully connected units, using the same activation function and regularization settings; The third hidden layer is a 512-dimensional fully connected unit, which continues to use Swish activation and Dropout with a dropout rate of 0.2; The fourth hidden layer consists of 256-dimensional fully connected units, maintaining the same activation and regularization strategies; The fifth hidden layer consists of 128-dimensional fully connected units, with consistent activation and regularization settings; The network ends with a 6-dimensional output layer, used to characterize the physical quantity to be determined; The network incorporates three types of physical constraints: the heat conduction equation of wet friction elements, the S-shaped variation law of contact pressure, and the friction torque transmission model.

[0079] In the above embodiments, the hybrid loss function, consisting of data loss, physical loss, and cross-modal fusion loss, is as follows:

[0080] In the formula, Total loss; This results in a loss of data consistency. For fusion loss; This represents the loss due to physical constraints.

[0081] In the above embodiments, a hybrid loss function consisting of data loss, physical loss, and cross-modal fusion loss is used to perform cross-model collaborative optimization of the MLLM multimodal feature fusion model and the PINN physical constraint model, including three stages: During training epochs 1–50, the main parameters of PINN are fixed, and only the LoRA adaptation layer of MLLM is updated to minimize the fusion loss. Loss of data consistency ; During training epochs 51–150, the MLLM parameters are kept constant, and only the gradients of PINN are updated to optimize the physical constraint loss. and data consistency loss ; In training epochs 151–200, parameter freezing was removed, and MLLM and PINN were trained simultaneously to optimize the total loss function. ; During training, the model checkpoint is saved every set number of rounds, and the model with the minimum total loss in the validation set is used as the final output after all training is completed.

[0082] In the above embodiments, based on the trained MLLM multimodal feature fusion model and PINN physical constraint model, the physical prediction results of PINN and the semantic reasoning ability of the large language model are obtained to generate a structured diagnostic report of the wet friction element, including: The acquired temperature data, vibration signals and operating condition text are preprocessed to generate temperature field images, vibration time-frequency diagrams and corresponding text vectors. Cross-modal representations are constructed using MLLM, and fused features are output. PINN then infers fault-related physical parameters based on these features. MLLM is then matched with a structured knowledge base to establish a mapping relationship between physical parameter deviations and potential fault causes, and finally a structured diagnostic report is generated and sent back to the operation and maintenance terminal. The diagnostic report includes the diagnosis time, operating parameters, fault type and confidence level, degree of damage, deviation of key physical parameters, fault cause analysis, and short-term and long-term maintenance recommendations and re-inspection cycle.

[0083] The system provided in this embodiment is used to execute the above-described method embodiments. For specific processes and details, please refer to the above embodiments, which will not be repeated here.

[0084] In one embodiment of the present invention, a computing device is provided. This computing device can be a terminal and may include a processor, a communication interface, memory, a display screen, and an input device. The processor, communication interface, and memory communicate with each other via a communication bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and computer programs. When the computer programs are executed by the processor, they implement the methods described in the above embodiments. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The communication interface is used for wired or wireless communication with external terminals. Wireless communication can be achieved through Wi-Fi, a management network, NFC (Near Field Communication), or other technologies. The display screen can be a liquid crystal display (LCD) or an e-ink display. The input device can be a touch layer covering the display screen, or buttons, a trackball, or a touchpad mounted on the casing of the computing device, or an external keyboard, touchpad, or mouse. The processor can call logical instructions stored in the memory.

[0085] Furthermore, the logical instructions in the aforementioned memory can be implemented as software functional units and sold or used as independent products, and can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0086] In one embodiment of the present invention, a computer program product is provided, the computer program product including a computer program stored on a non-transitory computer-readable storage medium, the computer program including program instructions, and when the program instructions are executed by a computer, the computer is able to perform the methods provided in the above-described method embodiments.

[0087] In one embodiment of the present invention, a non-transitory computer-readable storage medium is provided, which stores server instructions that cause a computer to perform the methods provided in the above embodiments.

[0088] The computer-readable storage medium provided in the above embodiments has a similar implementation principle and technical effect to the above method embodiments, and will not be described again here.

[0089] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0090] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0091] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0092] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A fault diagnosis method for wet friction elements based on MLLM-PINN, characterized in that, include: Three types of basic data—temperature, vibration, and text—are acquired from wet friction elements. After preprocessing these three types of basic data, they are input into the MLLM multimodal feature fusion model. Implicit correlations are obtained through a visual and semantic alignment mechanism, forming a high-dimensional fusion feature that includes visual, dynamic, and semantic information to enhance the completeness of fault feature expression. A PINN physical constraint model consisting of five fully connected hidden layers is constructed using PINN. High-dimensional fusion features are input into this PINN physical constraint model to output fault category, physical parameter deviation and damage degree, so as to achieve reliable constraint of physical mechanism on the diagnostic process. A hybrid loss function consisting of data loss, physical loss, and cross-modal fusion loss is used to perform cross-model collaborative optimization of the MLLM multimodal feature fusion model and the PINN physical constraint model. The Adam optimizer is used to optimize the model parameters, and the learning rate annealing strategy is combined with iteration to obtain the trained and optimized MLLM multimodal feature fusion model and PINN physical constraint model. Based on the trained MLLM multimodal feature fusion model and PINN physical constraint model, the physical prediction results of PINN and the semantic reasoning ability of the large language model are obtained to generate a structured diagnostic report of wet friction elements.

2. The method for fault diagnosis of wet friction elements based on MLLM-PINN as described in claim 1, characterized in that, Acquire three types of basic data for wet friction elements: temperature, vibration, and text data. Preprocess these three types of data, including: After obtaining temperature information through silicon carbide crystal thermometry, the Unet network is used to reconstruct the field of local point data to form a complete temperature field image. Vibration signals are acquired by sensors, and time-frequency feature maps are extracted through wavelet packet transform; The text data consists of operation and maintenance records, fault manuals, and expert experience, and is encoded into vector features using the BERT model.

3. The method for fault diagnosis of wet friction elements based on MLLM-PINN as described in claim 1, characterized in that, The input MLLM multimodal feature fusion model obtains implicit relationships through visual and semantic alignment mechanisms, forming high-dimensional fused features that contain visual, dynamic, and semantic information, including: A multimodal large language model structure is adopted, based on DiagLLM, and combined with LoRA low-rank adaptation technology to achieve parameter fine-tuning; In the fusion stage, the visual encoder of DiagLLM receives the time-frequency feature map obtained by converting the temperature field image and vibration signal, the text encoder processes the vector features, and the alignment and association of multimodal features are realized through the cross-attention mechanism to generate a high-dimensional fused feature vector. Consistency verification is performed by calculating the cosine similarity between the fused features and the single-modal features, and the similarity of each feature is required to be no less than the set threshold.

4. The method for fault diagnosis of wet friction elements based on MLLM-PINN as described in claim 1, characterized in that, A PINN physical constraint model consisting of five fully connected hidden layers is constructed using PINN, including: Input layer; The first hidden layer consists of 128-dimensional fully connected units, using the Swish activation function and configured with a Dropout regularization with a dropout rate of 0.2; The second hidden layer consists of 256-dimensional fully connected units, using the same activation function and regularization settings; The third hidden layer is a 512-dimensional fully connected unit, which continues to use Swish activation and Dropout with a dropout rate of 0.2; The fourth hidden layer consists of 256-dimensional fully connected units, maintaining the same activation and regularization strategies; The fifth hidden layer consists of 128-dimensional fully connected units, with consistent activation and regularization settings; The network ends with a 6-dimensional output layer, used to characterize the physical quantity to be determined; The network incorporates three types of physical constraints: the heat conduction equation of wet friction elements, the S-shaped variation law of contact pressure, and the friction torque transmission model.

5. The method for fault diagnosis of wet friction elements based on MLLM-PINN as described in claim 1, characterized in that, The hybrid loss function, consisting of data loss, physical loss, and cross-modal fusion loss, is as follows: In the formula, Total loss; This results in a loss of data consistency. For fusion loss; This represents the loss due to physical constraints.

6. The method for fault diagnosis of wet friction elements based on MLLM-PINN as described in claim 5, characterized in that, A hybrid loss function, consisting of data loss, physical loss, and cross-modal fusion loss, is used to perform cross-model co-optimization of the MLLM multimodal feature fusion model and the PINN physical constraint model, comprising three stages: During training epochs 1–50, the main parameters of PINN are fixed, and only the LoRA adaptation layer of MLLM is updated to minimize the fusion loss. Loss of data consistency ; During training epochs 51–150, the MLLM parameters are kept constant, and only the gradients of PINN are updated to optimize the physical constraint loss. and data consistency loss ; In training epochs 151–200, parameter freezing was removed, and MLLM and PINN were trained simultaneously to optimize the total loss function. ; During training, the model checkpoint is saved every set number of rounds, and the model with the minimum total loss in the validation set is used as the final output after all training is completed.

7. The method for fault diagnosis of wet friction elements based on MLLM-PINN as described in claim 1, characterized in that, Based on the trained MLLM multimodal feature fusion model and PINN physical constraint model, the physical prediction results of PINN and the semantic reasoning ability of the large language model are obtained to generate a structured diagnostic report of wet friction elements, including: The acquired temperature data, vibration signals and operating condition text are preprocessed to generate temperature field images, vibration time-frequency diagrams and corresponding text vectors. Cross-modal representations are constructed using MLLM, and fused features are output. PINN then infers fault-related physical parameters based on these features. MLLM is then matched with a structured knowledge base to establish a mapping relationship between physical parameter deviations and potential fault causes, and finally a structured diagnostic report is generated and sent back to the operation and maintenance terminal. The diagnostic report includes the diagnosis time, operating parameters, fault type and confidence level, degree of damage, deviation of key physical parameters, fault cause analysis, and short-term and long-term maintenance recommendations and re-inspection cycle.

8. A fault diagnosis method for wet friction elements based on MLLM-PINN, characterized in that, include: The data preprocessing and cross-modal feature fusion module acquires three basic data types of wet friction elements: temperature, vibration, and text. After preprocessing these three types of basic data, the data is input into the MLLM multimodal feature fusion model. Implicit correlations are obtained through a visual and semantic alignment mechanism, forming a high-dimensional fusion feature that includes visual, dynamic, and semantic information to enhance the completeness of fault feature expression. The physical constraint modeling module uses PINN to construct a PINN physical constraint model consisting of five fully connected hidden layers. High-dimensional fusion features are input into this PINN physical constraint model, and the fault category, physical parameter deviation, and damage degree are output to achieve reliable constraints on the physical mechanism during the diagnostic process. The cross-model collaborative optimization module uses a hybrid loss function consisting of data loss, physical loss, and cross-modal fusion loss to perform cross-model collaborative optimization of the MLLM multimodal feature fusion model and the PINN physical constraint model. It also uses the Adam optimizer to optimize the model parameters and combines the learning rate annealing strategy for iterative optimization to obtain the trained and optimized MLLM multimodal feature fusion model and PINN physical constraint model. The diagnostic output module, based on the trained MLLM multimodal feature fusion model and PINN physical constraint model, obtains the physical prediction results of PINN and the semantic reasoning ability of the large language model, and generates a structured diagnostic report of wet friction elements.

9. A computer-readable storage medium for storing one or more programs, characterized in that, The one or more programs include instructions that, when executed by a computing device, cause the computing device to perform any of the methods described in claims 1 to 7.

10. A computing device, characterized in that, include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for performing any of the methods described in claims 1 to 7.

Images