High-power heavy-duty gearbox transmission device fault diagnosis method based on multi-mode deep learning

Abstract

The invention provides a high-power heavy-duty gearbox transmission device fault diagnosis method based on multi-mode deep learning, and belongs to the technical field of gearbox fault diagnosis of deep learning. The method comprises the following steps: firstly, deploying a multi-modal sensor array at a key measuring point of equipment, and constructing a multi-modal high-fidelity special data set suitable for a high-power heavy-load gearbox scene; secondly, constructing a graph neural network for dynamically coupling equipment physical topology and complex environment influence to perceive propagation characteristics of a fault in a spatial dimension, and capturing a slow evolution rule of the fault in a time dimension in combination with a time Transform network; further, a collaborative attention fusion mechanism is designed, and cross-modal deep fusion is realized; and finally, through the trained optimal diagnosis model, a clear fault type and a confidence score are output. According to the method, high-precision recognition of early weak faults under the heavy load working condition is achieved, and the dynamic prediction and early warning capacity of latent faults is remarkably enhanced.

Description

Technical Field

[0001] This invention belongs to the field of gearbox fault diagnosis technology based on deep learning, and particularly relates to a fault diagnosis method for high-power heavy-duty gearbox transmission devices based on multimodal deep learning. Background Technology

[0002] High-power, heavy-duty gearbox transmissions are core components in modern industrial equipment power systems, widely used in extreme operating conditions such as ocean-going vessel propulsion systems, offshore drilling platforms, long-distance pipeline booster units, high-power compressors, and wind tunnel drive devices. These transmissions require long-term continuous operation under ultra-high torque density, extreme temperature variations, complex impact loads, and highly polluted environments. Their operational reliability directly determines the overall system's mission completion, economy, and safety. Under these extreme and complex conditions, traditional mechanical damage modes are significantly amplified, even giving rise to complex failure mechanisms unique to heavy-duty gearboxes. Examples include: cascading expansion of micro- and macro-pitting corrosion on gear meshing surfaces; the transformation of subsurface plasticity to brittleness under high load conditions, leading to deep crack propagation; synergistic wear of high-speed bearing outer rings due to creep and creep; and fatigue-loosening composite failures induced by coupled resonance between the gearbox housing and shaft system. A sudden failure of such a large gearbox could lead to major accidents such as ship stall, platform shutdown, and long-distance pipeline outages, resulting in economic losses in the tens of millions or even higher. Therefore, there is an urgent need to develop a set of fault diagnosis technologies for high-power, heavy-duty gearboxes that can perform real-time, reliable, and high-precision fault diagnosis.

[0003] Existing fault diagnosis methods for high-power, heavy-duty gearboxes can be broadly classified into three categories: traditional methods based on signal processing, methods based on conventional machine learning, and methods based on single-modal deep learning.

[0004] (1) Traditional signal processing-based methods: These methods rely on expert experience to perform time-domain / frequency-domain / time-frequency-domain analysis on vibration, oil, or torque signals, extracting statistical features such as envelope spectrum, cepstral spectrum, and energy entropy, and combining this with human experience to determine the fault type. When the gearbox is running on a test bench with constant load and low impact, this type of method can identify obvious cracks or broken teeth. However, under actual working conditions with impact load and multiple sources of strong noise, the signal exhibits strong non-stationary and strong impact characteristics. The weak impacts generated by early micro-pitting and deep peeling are often drowned out by environmental noise, operating noise, structural resonance, etc., making it very easy to miss early faults.

[0005] (2) Methods based on conventional machine learning: Usually, a large number of handcrafted features are extracted first, and then classification or linear regression is performed through SVM, random forest, and shallow networks. Its performance is heavily dependent on the characteristics of the feature engineering signal, which is highly non-stationary, non-Gaussian, and nonlinear, and it is difficult to generalize between different working conditions; at the same time, shallow models cannot explore the nonlinear coupling between multiple sensors and multiple components; and this type of method is extremely sensitive to changes in load, temperature, and lubrication status.

[0006] (3) Single-modal deep learning-based methods: In recent years, end-to-end deep learning models, represented by convolutional neural networks and recurrent neural networks, can directly learn deep representations from the original vibration waveforms, eliminating the need for manual features. However, most current research is limited to a single vibration mode, ignoring other information that is most valuable for diagnosis in heavy-duty gearboxes, such as high-frequency torque pulsation, housing temperature gradient, oil temperature, and instantaneous motor current. Relying solely on a single mode, the model struggles to distinguish between normal operating conditions and fault impacts, easily misjudging normal load impacts as faults (false alarms) or missing deep peeling as operating disturbances.

[0007] The above methods reveal four common shortcomings when facing high-power, heavy-load, and extremely variable scenarios: (1) The coupling between operating conditions and damage is difficult to decouple: the signal fluctuation of heavy-duty gearboxes is affected by strong disturbances such as load, temperature, lubrication, and contamination. Existing methods lack an adaptive decoupling mechanism for operating conditions, making it impossible to determine whether the abnormality comes from a sudden change in external operating conditions or from the initiation of an internal fault; (2) Lack of spatiotemporal evolution mechanism: The coupled dynamics of gear-bearing-box exhibit a progressive evolution relationship involving multiple components, multiple scales, and multiple stages. However, the existing "snapshot" diagnostics treat each sensor as an independent source of information, ignoring the force transmission path, structural topology, and the spatiotemporal path of gradual damage evolution, resulting in latent faults such as deep peeling and outer ring creep being detected too late; (3) The fusion of multi-source heterogeneous information is shallow and lacks semantic alignment: simple splicing or weighted fusion is difficult to establish deep semantic alignment between micro-vibration transients, macro-working conditions and material performance degradation, resulting in low information utilization and excessively wide lifetime prediction confidence intervals; (4) The model architecture lacks interpretability due to subjectivity: The architecture design of the diagnostic model relies heavily on the experience and trial and error of researchers, and lacks an objective and systematic method to automatically find the optimal model architecture for high-power heavy-load scenarios. Summary of the Invention

[0008] To address the above problems, this invention proposes a fault diagnosis method for high-power, heavy-duty gearbox transmission devices based on multimodal deep learning, comprising the following steps: S1, based on a multimodal sensor array, collects real-time data on gearbox operating status and gearbox working conditions; S2, Based on the physical topology of the gearbox, a dynamic weighted sensor topology graph containing force transmission path weights is constructed. Vibration signal data and noise signal data are transformed into dynamic graph node feature sequences that can describe gear meshing impact characteristics, local damage characteristics and energy distribution laws. The macroscopic operating state of the gearbox and the lubrication system state are transformed into contextual operating condition feature sequences that are synchronized with the node time-frequency characteristics. S3 inputs the dynamic weighted sensor topology map and dynamic graph node feature sequence output from S2 into the spatial correlation perception module, outputting a spatial fusion feature sequence; inputs the spatial fusion feature sequence into the spatiotemporal dynamic evolution module to obtain the spatiotemporal history feature sequence; inputs the spatiotemporal history feature sequence combined with the contextual condition feature sequence obtained from S2 into the dual-stream cross-attention network module to obtain the final high-dimensional fusion diagnostic vector; S4 takes the high-dimensional fusion diagnostic vector obtained in S3 and inputs it into the trained fault prediction learning model, outputting the fault category and confidence level.

[0009] Preferably, the gearbox operating status data includes vibration signal data, noise signal data, ambient temperature time series data, oil temperature time series data, oil pressure time series data, bearing speed time series data, and bearing torque time series data; The gearbox operating data includes bearing load rating, gearbox input power, gearbox output power, gear meshing frequency, lubricating oil flow rate, and oil pump pressure.

[0010] Preferably, in step S2, firstly, a dynamic sensor topology graph containing force transmission path weights is constructed based on the physical topology of the gearbox to express the mechanical coupling relationship between sensors. Fluctuations in bearing load level, bearing speed time series data, bearing torque time series data, and ambient temperature time series data will dynamically adjust the edge weights of the graph structure to reflect changes in the power path. Subsequently, vibration signal data and noise signal data are transformed into time-frequency feature sequences that can describe gear meshing impact characteristics, local damage characteristics, and energy distribution patterns through sliding window, short-time Fourier transform, and wavelet packet decomposition methods. Simultaneously, oil temperature time series data, oil pressure time series data, as well as low-frequency signals of gearbox input power, gearbox output power, gear meshing frequency, lubricating oil flow rate, and oil pump pressure are input into a multilayer perceptron to generate a context feature vector sequence describing the macroscopic health state and lubrication conditions. Finally, the output includes a dynamically weighted sensor topology graph reflecting dynamic coupling, a dynamic graph node feature sequence reflecting local damage to gears / bearings, and a context condition feature sequence describing macroscopic conditions of oil temperature and oil pressure fluctuations.

[0011] Preferably, the specific process for obtaining the dynamic weighted sensor topology map is as follows: First, based on the obtained 3D sensor layout diagram, all torque sensors, oil sensors, speed sensors, and temperature sensors are mapped as independent nodes in the graph structure. Second, based on the gearbox structural drawings and dynamic transmission chain, initial static edge connections are established according to gear meshing relationships, bearing supports, and force transmission paths. This is abstracted into an undirected weighted graph with node characteristics, forming the basic topology, formalized as follows: ; The high-frequency vibration sensor, acoustic sensor, oil sensor, speed sensor, and torque sensor are regarded as independent nodes, and the node set V represents the sensor position. Node features It includes real-time sensed multimodal data vectors; torque node features include instantaneous torque and load fluctuation rate; temperature node features include real-time oil temperature and temperature rise rate; pressure node features include absolute oil pressure and oil pressure fluctuation rate; speed node features include speed and speed fluctuation rate; vibration node features include impact characteristic index and fault modulation envelope energy. The static boundary is constructed based on the inherent power transmission and medium transmission relationships determined by the mechanical structure of the gearbox. This includes the power transmission side, which represents the torque and speed propagation path caused by gear meshing, shaft connection, and transmission stage arrangement. The medium transmission relationship is determined by the fluid path. edge weight This represents the coupling strength between two nodes. The node edges adopt the idea of ​​static connection and dynamic weight. The initial value of the static edge weight is a uniform value of 1, which is used to determine the connectivity graph, so that the dynamic weight update module can start learning from the unbiased initial state. Simultaneously, a load-driven dynamic edge weight update mechanism is constructed. Based on vibration signal data, bearing load level, bearing torque time series data, oil temperature time series data, oil pressure time series data, and bearing speed time series data, a multi-source coupling model is built, including a load effect model, a temperature effect model, and an oil pressure effect model, to calculate the adjustment coefficients of each mode.

[0012] Preferably, the specific process for obtaining the dynamic graph node feature sequence is as follows: The input consists of structural vibration signals and acoustic noise signals. First, a sliding window mechanism is used to uniformly divide the long-sequence vibration and noise signals into time segments. By setting a fixed window length and window overlap rate, the continuous time-domain waveform is strictly divided into a series of short time segments aligned on the time axis, ensuring that all sensor signals form corresponding window segments at the same time step. Second, a Short-Time Fourier Transform (STFT) is performed on each time segment to obtain its two-dimensional time-frequency spectrum, reflecting the change of energy distribution at different frequencies over time. Third, each two-dimensional time-frequency spectrum is transformed into a fixed-length one-dimensional feature vector through frequency domain pooling, energy bandwidth aggregation, or convolution feature compression, making it a snapshot of the time-frequency features of the corresponding sensor node at that time step. Finally, a dynamic graph node feature sequence is output, organized as a list sorted by time, with each time step containing a set of time-frequency feature vectors for all sensor nodes.

[0013] Preferably, the specific process for obtaining the contextual condition feature sequence is as follows: The inputs include time-series data of oil temperature, oil pressure, oil flow rate, oil pump pressure, and operating parameters such as input power, output power, and gear meshing frequency. First, to achieve cross-modal time alignment, a sliding window mechanism is used to synchronously segment the original time series of all operating condition data. The multi-dimensional operating condition signal is divided into a series of time segments, ensuring that each time segment corresponds on the time axis to all sensor signal time segments acquired in the dynamic graph node feature sequence acquisition, achieving time-step-level uniformity. Second, within each time segment, statistical contextual features are extracted using an independent operating condition feature extraction network. The features include the average oil temperature and its rate of change, the average oil pressure and its pulsation amplitude, the oil flow stability factor, the average oil pump pressure, the load factor, and the meshing frequency characteristics, which serve as descriptors characterizing changes in macroscopic operating conditions. The operating condition feature extraction network consists of an input layer, multiple ReLU-activated nonlinear hidden layers, and an output layer, used to capture the nonlinear coupling and multivariate interaction relationships between operating condition parameters. Finally, it outputs a contextual operating condition feature sequence, which is strictly synchronized with the dynamic graph node feature sequence in terms of time steps, so that each time point has both the microscopic time-frequency characteristics from the sensor and the macroscopic system state from the operating conditions.

[0014] Preferably, the spatial correlation sensing module specifically comprises: A graph attention module (GAT) is constructed, consisting of three stacked graph attention layers. Each layer is a multi-head graph attention sublayer with residual connections and layer normalization. Graph attention network inference is performed independently for each time step in the sequence. For any time step, GAT takes the dynamic graph node features at the current time as input and aggregates neighborhood features based on the weighted adjacency matrix at that time. Through a self-attention mechanism, GAT adaptively calculates attention coefficients based on the correlation of features of adjacent nodes in the topology graph, ensuring that edge weights not only reflect device structural dependencies. Subsequently, nodes update their representations based on weighted fusion of edge weights, realizing spatial modeling of device structural response. The final output is a spatially fused feature sequence. Each element in the sequence corresponds to a time step and contains the spatial features of all nodes after GAT aggregation, used to describe the overall structural response pattern of the gearbox.

[0015] Preferably, the spatiotemporal dynamic evolution module is a multi-layered stacked temporal Transformer encoder, with each encoder block containing a multi-head self-attention mechanism and a feedforward neural network. The multi-head self-attention mechanism is used to calculate the dependency between any two time steps globally, enabling the model to simultaneously capture short-term transient changes, including high-frequency impacts caused by oil film rupture, as well as long-term gradual trends, including the evolution of gear pitting, the long-term accumulation of fatigue spalling in bearing raceways, and the impact of oil film degradation on vibration energy distribution. The feedforward neural network is used to nonlinearly enhance the output of the self-attention mechanism. All sub-layers contain residual connections and layer normalization. Finally, a spatiotemporal history feature sequence is output, in which each time step encodes the spatial aggregation features of the current moment and also incorporates the dynamic evolution information of its preceding moments.

[0016] Preferably, the dual-stream cross-attention network module specifically comprises: First, using the contextual operating condition feature sequence as the query term, an attention distribution is calculated for the spatiotemporal historical feature sequence, enabling the macroscopic operating condition to selectively focus on the microstructural changes that have the most significant impact. The reverse attention path uses the spatiotemporal historical feature sequence as the query term and performs reverse attention on the contextual operating condition feature sequence, enabling the microstructural dynamics to actively extract the key operating condition variables on which they depend. The outputs of the bidirectional cross-attention are passed through their respective feedforward fully connected networks, and the two sequences are updated using residual connections and layer normalization to provide input for the next layer of fusion. As the multi-layer collaborative fusion progresses layer by layer, the information of the two branches is deeply coupled at different levels of abstraction. Finally, at the end of the fusion network, the output vector corresponding to the position of the classification embedding vector is extracted to obtain the final high-dimensional fusion diagnostic vector. This vector comprehensively encodes the gearbox structural topological features, the dynamic evolution law across time scales, the vibration and acoustic micro-response modes, operating conditions, oil health, and load trends.

[0017] Preferably, the fault prediction learning model employs a three-layer linear mapping to map the high-dimensional fused diagnostic vector to the fault category space and outputs the fault category probability distribution.

[0018] Compared with the prior art, the present invention has the following beneficial effects: (1) Achieving high-precision identification of early and weak faults under heavy-load conditions: This invention, through multimodal deep fusion, condition-adaptive modeling, and dynamic topology graph structure, enables the model to effectively isolate weak early damage features that are usually submerged under real heavy-load conditions with strong impact, strong noise, and strong non-stationarity. Compared with traditional vibration diagnosis, this invention significantly improves the ability to extract weak signals of early faults and the diagnostic sensitivity, and significantly reduces the false negative rate; (2) Significantly enhances the dynamic prediction and early warning capability of latent faults: For the first time, this invention dynamically maps the force flow topology, boundary conditions and load paths of heavy-duty gearboxes into a spatiotemporal diagram structure, and models the evolution chain of faults in depth through time Transformer, enabling the system to capture and quantify the slow accumulation process of deep peeling, the cascade expansion of pitting to macro-pitting, and the progressive damage process induced by bearing outer ring creep and frictional heat. This breaks through the limitation of traditional snapshot-type diagnosis that cannot identify long-term evolution characteristics, enabling latent, weakly gradual, and long-cycle faults to be warned in advance, significantly improving the long-term safety and predictability of heavy-duty equipment operation. (3) Provides comprehensive and highly reliable intelligent diagnostic decision-making: the diagnosis is upgraded from relying on single vibration information to deep collaborative reasoning of multi-source information, the reliability of the output diagnostic results is significantly improved, and finally provides more interpretable and more credible intelligent diagnostic basis for the operation and maintenance of heavy-duty equipment. (4) Using automated machine learning to achieve objective optimal configuration and full-process traceability of diagnostic models: This invention introduces differentiable architecture search automated machine learning technology to systematically optimize the model architecture, get rid of the subjective influence of manual trial and error on architecture design, automatically search for the optimal structure from a large-scale candidate space, and can achieve adaptive reconstruction of the model for different equipment and different load scenarios. The output results are reproducible and traceable, and are applicable to different engineering sites. Attached Figure Description

[0019] Figure 1 This is a flowchart illustrating the overall technical route of the present invention.

[0020] Figure 2 This includes a layout diagram of the sensor's physical location and a diagnostic flowchart.

[0021] Figure 3 A general flowchart is constructed to consider the impact of heavy-load dynamics on data processing and multi-view features.

[0022] Figure 4 This is a diagram showing the overall architecture of a fault diagnosis model based on graph attention and collaborative fusion Transformer.

[0023] Figure 5 The diagram shows the confusion matrix of model performance in the example.

[0024] Figure 6 This is a bar chart comparing the advancement of the models in the examples.

[0025] Figure 7 This is an example of noise robustness analysis of the model in the embodiment. Detailed Implementation

[0026] This invention proposes a fault diagnosis method for high-power, heavy-duty gearbox transmission devices based on multimodal deep learning. The overall process is as follows: Figure 1 As shown: S1. Construction of a Multimodal High-Fidelity Dataset for Heavy-Duty Operations: The core task of this step is to construct a dedicated multimodal high-fidelity dataset suitable for high-power, heavy-duty gearbox scenarios. This process first involves deploying a multimodal sensor array—comprising high-frequency vibration sensors, acoustic sensors, oil sensors, and speed / torque sensors—at key locations in the input shaft, output shaft, meshing area, bearing housing, and lubrication system, based on the gearbox's load characteristics, to simultaneously collect multi-source operating status information. Secondly, considering the sudden loading and operating condition fluctuations of heavy-duty equipment, operating condition types covering different load levels, lubrication states, and fault types are constructed, including pitting, spalling, fatigue cracks, galling, and early bearing damage. The multi-source time-series data is then time-aligned, labeled, and bound to operating condition metadata according to a predefined fault mode library. The final result is a structured multimodal data sample library of heavy-duty gearboxes. Each sample contains seven types of gearbox operating state data, including vibration signal data, noise signal data, ambient temperature time series data, oil temperature time series data, oil pressure time series data, bearing speed time series data, and bearing torque time series data, as well as six types of gearbox operating condition data, including bearing load level, gearbox input power, gearbox output power, gear meshing frequency, lubricating oil flow rate, and oil pump pressure. Each sample also has a gearbox health status label, providing a high-fidelity data foundation for subsequent deep diagnostic model training and validation. S2. Multimodal Data Processing and Multi-View Feature Construction Module Considering Heavy-Load Dynamics: This step aims to transform the original multi-source heterogeneous signals into multi-view structured features that contain actual mechanical meaning and reflect the dynamic evolution law. First, this invention constructs a dynamic sensor topology graph containing force transmission path weights based on the physical topology of the gearbox, used to express the mechanical coupling relationship between sensors. Fluctuations in bearing load level, bearing speed time-series data, bearing torque time-series data, and ambient temperature time-series data dynamically adjust the edge weights of the graph structure to reflect changes in the power path. Subsequently, using sliding window, short-time Fourier transform, and wavelet packet decomposition methods, vibration signal data and noise signal data are transformed into time-frequency feature sequences that can describe gear meshing impact characteristics, local damage characteristics, and energy distribution laws. Simultaneously, oil temperature time-series data, oil pressure time-series data, and low-frequency signals such as gearbox input power, gearbox output power, gear meshing frequency, lubricating oil flow rate, and oil pump pressure are input into a multilayer perceptron to generate a context feature vector sequence describing the macroscopic health state and lubrication conditions. This step ultimately provides three key inputs for subsequent models: a dynamic graph structure that reflects the dynamic coupling, a microscopic time-frequency feature sequence that reflects local damage to gears / bearings, and a contextual feature sequence that describes the macroscopic working conditions of oil temperature and oil pressure fluctuations. S3. Construction of a Heavy-Duty Gearbox Fault Diagnosis Model Based on Graph Attention and Collaborative Fusion Transformer: This step is the core of fault diagnosis, aiming to achieve accurate diagnosis under complex operating conditions through a multi-stage deep learning network. First, high-frequency sensor signals are mapped onto the aforementioned topology graph and input into a graph attention network. The coupling relationships between different sensors are fused in the spatial dimension to obtain a graph-level feature sequence reflecting the overall dynamic state of the gearbox. Next, the above spatially fused features are input into a temporal Transformer encoder to capture the long-term evolution patterns of pitting expansion, bearing degradation, and lubrication deterioration in the time dimension, thereby achieving state trend modeling. Finally, through a collaborative attention fusion network, the power path changes containing load level, bearing speed time series data, and bearing torque time series data, the time-frequency signal feature sequence of specific acoustic patterns containing vibration signal data and noise signal data, are deeply interacted and fused with the oil state containing oil temperature time series data and oil pressure time series data, and the context features of the macroscopic working condition background containing gearbox input power, gearbox output power, gear meshing frequency, lubricating oil flow rate, and oil pump pressure. This automatically eliminates the interference of working condition fluctuations, extracts the essential features of the fault, and obtains the final high-dimensional fusion diagnostic vector. This diagnostic vector can be used to realize fault type identification and health index assessment. S4. Automated Machine Learning Model Optimization Module for Industrial Deployment: This step aims to build an automated model optimization process for large-scale industrial equipment to obtain a final diagnostic model with optimal performance that meets the requirements of actual engineering deployment. First, the entire heavy-duty gearbox diagnostic network is parameterized, constructing a multi-dimensional search space including model depth, graph network structure, number of Transformer layers, number of attention heads, and convolutional kernel size. Then, a differentiable neural architecture search technique is used to automatically search for the optimal architecture within this space. The candidate models obtained from the search will undergo multi-dimensional performance verification, including classification accuracy, robustness, latency, and complexity, to ensure they meet all preset indicators. Finally, the verified architecture will undergo production-grade final training and solidification, producing a final diagnostic model file with a complete performance evaluation report that can be directly used for deployment and certification.

[0027] The specific implementation process of the present invention will be described in detail below with reference to specific embodiments.

[0028] S1. Construction of a multimodal high-fidelity dataset for heavy-load operating conditions This step aims to construct a multimodal high-fidelity dataset that can comprehensively reflect the load impact, dynamic response, lubrication fluctuations, and structural health status of high-power heavy-duty gearboxes in actual operation, providing a reliable training foundation for subsequent deep diagnostic models.

[0029] like Figure 2As shown, for the acquisition of multi-source heterogeneous information in gear transmission systems, this invention deploys multiple types of sensors in different structural parts of the gearbox for monitoring. Around the key force paths of the gear transmission chain, multiple types of industrial-grade sensors are deployed at key structural points such as the gear meshing area, the input and output shaft sections, bearing seats, and the gearbox support positions. Specifically, this includes: high-frequency structural vibration sensors positioned at the bottom of the gearbox and at the gearbox support locations to capture weak gear meshing impact energy; broadband acoustic sensors and acoustic emission probes positioned at the top of the gearbox and near the openings in the meshing area to detect abnormal impact sounds from the tooth surfaces, changes in meshing noise, and high-frequency acoustic signals generated by the rupture of the lubricating oil film; a speed sensor installed on the input shaft (long shaft end) to record speed information, and a dynamic torque sensor installed on the output shaft end (short shaft end) to record torque changes caused by load fluctuations, tooth surface wear, or transmission errors during power chain transmission; online sensors for oil temperature, oil pressure, oil viscosity, and particle size detection arranged at the inlet and outlet of the lubrication oil circuit to reflect real-time changes in lubrication system temperature rise, pressure fluctuations, and abrasive particle content; and a temperature sensor installed on the outer surface of the gearbox near the high-load gear meshing area to monitor local thermal evolution caused by friction, lubrication degradation, or thermal expansion. These sensors work together to form a real-time acquisition system for multimodal signals from the heavy-duty gearbox. Secondly, the industrial control system synchronously collects operating parameters related to mechanical load, including key variables reflecting the macroscopic working state of the equipment such as input and output power, load level, gear meshing frequency, lubricating oil flow rate and oil pump pressure.

[0030] Furthermore, typical failure modes were simulated and recorded during heavy-duty test bench and actual field operation, including gear failures, bearing failures, transmission chain failures, lubrication system failures, and structural failures. Gear failures included pitting, wear, tooth breakage, and meshing damage; bearing failures included inner ring, outer ring, rolling element, and cage failures; transmission chain failures included shaft misalignment, rotor imbalance, and shaft bending; lubrication system failures included oil deterioration, oil film degradation, abnormal oil temperature, and insufficient oil pressure; and structural failures included housing loosening, anchor bolt loosening, and unstable support structure.

[0031] Finally, all collected multi-source heterogeneous data were rigorously synchronized in time, segmented using a sliding window, and labeled with status tags to construct the final multimodal sample library. Each sample contains seven types of gearbox operating status data, including vibration signal data, noise signal data, ambient temperature time-series data, oil temperature time-series data, oil pressure time-series data, bearing speed time-series data, and bearing torque time-series data; and six types of gearbox operating condition data, including bearing load level, gearbox input power, gearbox output power, gear meshing frequency, lubricating oil flow rate, and oil pump pressure; as well as corresponding health status category tags, including normal state, gear faults, bearing faults, transmission chain faults, lubrication system faults, and structural faults.

[0032] S2. Data processing and multi-view feature construction considering the effects of heavy-load dynamics This step aims to transform the raw multi-source sensor data and operating parameters into structured, multi-view spatiotemporal features and contextual feature vector sequences suitable for deep model inference, and to construct a sensor topology that reflects the dynamic changes in the force transmission chain within the gearbox. The overall process includes three parts: sensor physical topology construction, node signal time-frequency embedding and serialization, and contextual embedding of operating parameters. Figure 3 As shown.

[0033] S2.1 Sensor physical topology construction: This step constructs a dynamically weighted sensor topology diagram that reflects the actual mechanical coupling relationship inside the gearbox by using dynamic data of load changes, torque fluctuations, and ambient temperature changes.

[0034] First, based on the 3D sensor layout diagram obtained in step S1, all torque sensors, oil sensors, speed sensors, and temperature sensors are mapped as independent nodes in the graph structure. Second, based on the gearbox structural drawings and dynamic transmission chain, initial static edge connections are established according to gear meshing relationships, bearing supports, and force transmission paths. This is abstracted as an undirected weighted graph with node characteristics, forming the basic topology, which can be formalized as: .

[0035] (1) Node ( V ) Based on the sensor 3D layout diagram in step S1, the high-frequency vibration sensor, acoustic sensor, oil sensor, speed sensor, and torque sensor are considered as independent nodes, and the node set V represents the sensor position.

[0036] (2) Node characteristics X ) Each node not only contains the sensor location, but also its real-time perceived multimodal data vector, constituting the node's features. Torque node features include instantaneous torque and load fluctuation rate; temperature node features include real-time oil temperature and temperature rise rate; pressure node features include absolute oil pressure and oil pressure fluctuation rate; speed node features include speed and speed fluctuation rate; vibration node features include impact characteristic index and fault modulation envelope energy. Node features can be uniformly represented as: .

[0037] (3) edge ( E ) The static edges are constructed based on the inherent power transmission and media transmission relationships determined by the gearbox's mechanical structure. This includes the power transmission edges, representing the torque and speed propagation paths caused by gear meshing, shaft connections, and the arrangement of transmission stages. The media transmission relationships are determined by the fluid path. The following are the sensor pairs used to establish the static edges: The torque sensor at the output shaft end and the speed sensor at the input shaft end, which are mechanically meshed, represent the torque / speed transmission between the shaft-end sensing units; the high-frequency structural vibration sensor at the gearbox support position and the acoustic sensor on the top of the gearbox represent the solid-air acoustic coupling and impact radiation path; the high-frequency structural vibration sensor and the temperature sensor in the high-load gear meshing area represent the propagation of vibration energy through the solid structure; the torque sensor at the output shaft end and the high-frequency structural vibration sensor for gear meshing impact energy represent the vibration response generated by load fluctuations through impact; A static edge is established between the oil sensor and the temperature sensor in the near-meshing zone to represent the influence of the medium on the local thermal and lubrication states; an edge is established between the oil sensor and the torque sensor to represent the influence of the lubrication state on the power transmission.

[0038] (4) Edge weights W ) Edge weights represent the coupling strength between two nodes. The node edges employ a combination of static connections and dynamic weights. The initial value of the static edge weights is a uniform 1, used to define the connectivity of the graph, allowing the dynamic weight update module to learn from an unbiased initial state. During system operation, the load-driven dynamic weight update module adjusts the weights in real time based on multimodal data, gradually converging the edge weights of different paths to values ​​reflecting the actual operating conditions. This ensures the effectiveness and accuracy of the graph model in health monitoring and fault diagnosis.

[0039] Secondly, and most importantly, this step involves constructing a load-driven dynamic edge weight update mechanism. This invention designs a load-driven dynamic weight function, based on a multi-source coupled model using vibration signal data, bearing load level, bearing torque time-series data, oil temperature time-series data, oil pressure time-series data, and bearing speed time-series data. This dynamic update mechanism includes load effect models, temperature effect models, and oil pressure effect models to calculate the adjustment coefficients for each mode.

[0040] S2.1.1 Load Effect Model The load effect model describes the impact of real-time torque fluctuations on gear meshing contact stiffness. As the load increases, the contact stiffness in the gear meshing zone changes, thus affecting the transmission path of the influence. The adjustment coefficients calculated by this model dynamically adjust the weights between sensors using the following strategy: Condition A: Torque fluctuation exceeds the set threshold (±10%). At this time, the load is large or changes abruptly, and the contact stiffness changes significantly, resulting in a redistribution of the meshing force transmission path.

[0041] Adjustment strategy: Increase the edge weights between corresponding sensors to reflect the stronger influence of the sensor on the force transmission path, and update the weights. ; in The adjustment coefficient is set to 0.4. When the torque deviation is 25%, An increase of approximately 10%, It is a reference torque, which is the median torque under the working conditions, and the upper and lower limits are trimmed to prevent excessive amplification or reduction.

[0042] If condition A is not met, i.e. the torque fluctuation is within ±10% and the load change is small, the stiffness change caused by the current load is considered to have a relatively weak impact. In this case, the weight is not increased or decreased, and the existing sensor weight is maintained.

[0043] S2.1.2 Temperature Effect Model The temperature effect model describes the process by which increasing oil temperature leads to a decrease in lubricating viscosity and changes in damping characteristics. As oil temperature rises, the viscosity of the lubricating oil decreases, affecting the formation of the lubricating film and thus altering the friction and damping parameters of the gears. The update strategy for the adjustment coefficients of this model is as follows: Condition B: Oil temperature variation exceeds ±10°C Significant temperature fluctuations lead to reduced lubricity, changes in the coefficient of friction, and affect contact damping characteristics. The adjustment strategy is to reduce the weights of lubrication-related sensors to simulate the energy loss and increased friction caused by poor lubrication. ; in, The adjustment coefficient is set to 0.15. If the temperature change exceeds 20°C, the weight reduction will exceed 10%. It is a reference temperature, taken from the median of healthy samples, and adjusted according to the range of oil temperature changes, with upper and lower limits cropped to prevent excessive magnification or reduction.

[0044] S2.1.3 Oil Pressure Effect Model The oil pressure effect model describes the impact of insufficient oil pressure or oil pressure fluctuations on the friction contact area. Insufficient oil pressure leads to enhanced friction contact, causing changes in local vibration energy coupling and resulting in greater energy loss.

[0045] Condition C: Oil pressure is below 80% of rated pressure. When the oil pressure is lower than the set standard, i.e., the oil pressure is insufficient, the friction increases significantly, the local contact energy increases, resulting in more energy loss, which needs to be reflected in the sensor's physical weight.

[0046] Adjustment strategy: Increase the side weights associated with the oil level monitoring sensor to reflect the impact of insufficient oil pressure on friction. The specific increase is determined by the percentage decrease in oil pressure, calculated using the following formula: ; in This is an adjustment coefficient, set to 0.5. When the oil pressure drops to 70%, the oil pressure adjustment coefficient increases by 15%. This is the reference oil pressure, taken from the system's calibrated normal oil pressure or the median of a healthy sample. Adjustments are made based on the range of oil pressure changes, with upper and lower limits cropped to prevent excessive amplification or reduction.

[0047] S2.1.4 Comprehensive Adjustment Strategy Based on the adjustment strategies of the three models mentioned above, the final edge weight update can be adjusted according to the combined impact of each effect. For example, if the load effect model and the oil pressure effect model are triggered simultaneously, and the oil pressure is insufficient while the load fluctuates significantly, the weights of the relevant sensors will be increased and decreased simultaneously. ; in, Each comes from its own dynamic adjustment calculation.

[0048] Through this multi-source coupled dynamic edge weight update mechanism, the present invention can accurately reflect the complex mechanical and thermal state of the gearbox transmission device under heavy load conditions, providing a solid data foundation for subsequent fault diagnosis and prediction.

[0049] S2.2 Time-frequency embedding and serialization of node signals: This step aims to transform the raw time-domain waveforms acquired by the sensors into a sequence of time-frequency feature vectors that can be processed by a graph neural network. The input consists of structural vibration signals and acoustic noise signals from the sample library in step S1. First, to capture the transient characteristics of the signals, this invention employs a sliding window mechanism to uniformly divide the long-sequence vibration and noise signals into time segments. By setting a fixed window length and window overlap rate, the continuous time-domain waveform is strictly divided into a series of short-time segments aligned on the time axis, ensuring that all sensor signals form corresponding window segments at the same time step. Second, a short-time Fourier transform (STFT) is performed on each time segment to obtain its two-dimensional time-frequency spectrum, reflecting the change of energy distribution at different frequencies over time. Third, to unify the input format of subsequent models, each two-dimensional time-frequency spectrum is transformed into a fixed-length one-dimensional feature vector through frequency domain pooling, energy bandwidth aggregation, or convolutional feature compression, making it a snapshot of the time-frequency features of the corresponding sensor node at that time step. This step ultimately outputs a dynamic graph node feature sequence. Its organization is a set of time-sorted lists, with each time step containing a set of time-frequency feature vectors of all sensor nodes, providing a unified time reference for multi-sensor input for subsequent dynamic graph convolution processing.

[0050] S2.3 Context embedding of operating condition parameters: This step aims to transform the macroscopic operating state of the gearbox and the lubrication system state into a contextual feature sequence synchronized with the node time-frequency characteristics. Inputs include oil temperature time-series data, oil pressure time-series data, oil flow rate, oil pump pressure, and operating parameters such as input power, output power, and gear meshing frequency. First, to achieve cross-modal time alignment, this invention employs a sliding window mechanism identical to step S2.2 to synchronously segment the original time series of all operating condition data. Specifically, multi-dimensional operating condition signals such as oil temperature, oil pressure, flow rate, and power are segmented into a series of time segments with the same window length and overlap rate, ensuring that each time segment strictly corresponds to the time segments of all sensor signals in step S2.2 on the time axis, achieving time-step-level uniformity. Second, statistical contextual features are extracted within each time segment, including descriptors characterizing macroscopic operating condition changes such as the mean and rate of change of oil temperature, the mean and pulsation amplitude of oil pressure, the oil flow stability factor, the average oil pump pressure, the load factor, and meshing frequency characteristics. Based on this, the present invention constructs an independent operating condition feature extraction network, which consists of an input layer, multiple ReLU-activated nonlinear hidden layers, and an output layer. This network captures the nonlinear coupling and multivariate interactions between operating condition parameters and maps multiple statistical features into low-dimensional, high-information-density contextual feature vectors. The final output of this step is a contextual operating condition feature sequence, which is strictly synchronized in time step with the dynamic graph node feature sequence output in step S2.2. This ensures that each time point possesses both microscopic time-frequency characteristics from the sensors and macroscopic system states from the operating conditions. This synchronized contextual sequence is then provided as input to the cross-modal deep fusion module in step S3.3 to achieve joint modeling of node-level signals and system-level operating conditions.

[0051] S3. Construction of a fault diagnosis model based on graph attention and collaborative fusion Transformer This step aims to construct a deep fusion diagnostic model capable of simultaneously modeling the spatial relationships, temporal dynamic evolution, and macroscopic operating conditions of the gearbox structure. The model consists of a graph attention network (GAT), a temporal Transformer encoder, and a cross-modal collaborative attention network. Through structured modeling of multi-source dynamic signals and operational background information, it generates a high-dimensional global fault diagnosis representation vector. The overall architecture of the model in this step is as follows: Figure 4 As shown.

[0052] S3.1 Spatial Correlation Perception Module: This step receives the dynamic graph node feature sequence from S2.2 and the dynamic weighted sensor topology graph from S2.1, which provides prior knowledge of the device structure as it changes with the environment. This step constructs a Graph Attention (GAT) module, consisting of three stacked graph attention layers. Each layer is a multi-head graph attention sublayer with residual connections and layer normalization. Graph attention network inference is performed independently for each time step in the sequence. For any given time step, GAT takes the dynamic graph node features at the current time as input and aggregates neighborhood features based on the weighted adjacency matrix at that time. Through a self-attention mechanism, GAT adaptively calculates attention coefficients based on the correlation of features between adjacent nodes in the topology graph, ensuring that edge weights reflect not only device structure dependence but also dynamic changes driven by load. Subsequently, nodes update their representations based on weighted fusion of edge weights, achieving spatial modeling of the device structure response. The final output of this step is a spatially fused feature sequence. Each element in the sequence corresponds to a time step and contains the spatial features of all nodes after GAT aggregation, used to describe the overall structural response pattern of the gearbox at that time. The result will be passed to step S3.2 for time series evolution analysis.

[0053] S3.2 Spatiotemporal Dynamic Evolution Module: The input to this step is the spatial fusion feature sequence generated in step S3.1, which is a sequence with the spatial dimension aggregated but the temporal dimension unmodeled. This spatial fusion feature sequence is input to a multi-layered, stacked temporal Transformer encoder. Each encoder block contains two parts: a multi-head self-attention mechanism and a feedforward neural network (FFN). The multi-head self-attention mechanism is used to calculate the dependencies between any two time steps globally, enabling the model to simultaneously capture short-term transient changes, including high-frequency shocks caused by oil film rupture, and long-term gradual trends, including the evolution of gear pitting, the long-term accumulation of bearing raceway fatigue spalling, and the impact of oil film degradation on vibration energy distribution. The feedforward neural network is used to nonlinearly enhance the output of the self-attention mechanism. All sub-layers contain residual connections and layer normalization to ensure the stability of deep network training. This step ultimately outputs a spatiotemporal history feature sequence. Each time step in this sequence not only encodes the spatial aggregated features of the current moment but also incorporates the dynamic evolution information of its preceding moments. This sequence will serve as one input to the cross-modal fusion module, providing a dynamic background of the microstructure.

[0054] S3.3 Dual-Stream Cross-Attention Network Module: This step receives two strictly synchronized input sequences: (1) The contextual operating condition feature sequence from S2.3 represents the macroscopic operating status and oil health status; (2) The spatiotemporal history feature sequence from S3.2 represents the temporal evolution of the gearbox structural response.

[0055] First, to construct a unified discrimination space, this step inserts a learnable classification embedding vector at the beginning of the spatiotemporal history sequence. This vector will serve as an information convergence point throughout the model. Subsequently, both sequences are input into a two-stream collaborative attention structure. Its core is the bidirectional cross-attention mechanism in each fusion block: First, using the contextual operating condition feature sequence as the query term, an attention distribution is calculated for the spatiotemporal historical feature sequence, enabling the macroscopic operating condition to selectively focus on the microscopic structural changes that have the most significant impact. The reverse attention path uses the spatiotemporal historical feature sequence as the query term and performs reverse attention on the contextual operating condition feature sequence, allowing the microscopic structural dynamics to proactively extract the key operating condition variables they depend on. The outputs of the bidirectional cross-attention are passed through their respective feedforward fully connected networks, and the two sequences are updated using residual connections and layer normalization, providing input for the next layer of fusion. As the multi-layer collaborative fusion progresses layer by layer, the information from the two branches is deeply coupled at different levels of abstraction. Finally, at the end of the fusion network, the output vector corresponding to the classification embedding vector position is extracted. This vector has continuously aggregated key discriminative information from microscopic structure, temporal dynamics, and macroscopic operating conditions throughout the inference process. This step outputs the final high-dimensional fusion diagnostic vector, which comprehensively encodes the gearbox structural topological features, dynamic evolution across time scales, vibration and acoustic micro-response modes, operating conditions, oil health, and load trends. This fusion vector will serve as the final judgment criterion and will be input into the fault prediction learning model in step S4.

[0056] S4. Training and Deployment of Fault Prediction Learning Model Based on the final fused diagnostic vector output in step S3, this step constructs a fault prediction learning model with high robustness, high generalization ability, and online deployment capability. The data processing module, collaborative fusion network, and prediction model are then integrated into the industrial monitoring environment to achieve online fault identification and health status prediction of high-power heavy-duty gearbox transmission devices.

[0057] Training and solidification of the S4.1 fault prediction learning model: This step, based on the high-dimensional fused diagnostic vector output from step S3, trains and solidifies the fault prediction learning model to achieve accurate classification of equipment health status. This step uses a linear classification head to receive the high-dimensional fused diagnostic vector output from S3, employing a three-layer linear mapping to map the high-dimensional features to the fault category space, outputting the fault category probability distribution. This structure ensures the model's inference speed and deployment stability in engineering applications. Standard cross-entropy loss is used for training. After training in this step, a solidified fault prediction learning model that can be directly used for deployment is obtained.

[0058] S4.2 model deployed to monitoring workstation: This step integrates and deploys the trained fault prediction learning model, data processing module, and collaborative fusion network into the online monitoring system of the target gearbox. This deployment environment is responsible for real-time data access, online feature calculation module, collaborative fusion network inference, and result output, ensuring that the diagnostic model can operate stably under complex heavy-load conditions and lubrication conditions with high load and strong mechanical impact.

[0059] S4.3 Real-time Data Stream Access: This step injects the multimodal sensor data from step S1 into the monitoring system in real time as real-time input for fault analysis. This includes seven types of gearbox operating status data: vibration signal data, noise signal data, ambient temperature time-series data, oil temperature time-series data, oil pressure time-series data, bearing speed time-series data, and bearing torque time-series data; and six types of gearbox operating condition data: bearing load level, gearbox input power, gearbox output power, gear meshing frequency, lubricating oil flow rate, and oil pump pressure. This data forms the basis for the system's real-time inference.

[0060] S4.4 Online Feature Processing, Fusion Inference, and Fault Detection: This step completes the entire real-time analysis process, including online feature extraction, graph structure fusion inference, and final fault identification. Online feature processing extracts dynamic graph node feature sequences and contextual operating condition feature sequences from the real-time input data. Collaborative fusion inference inputs these two types of sequences into the graph attention network designed in step S3 to obtain a spatial fusion feature sequence, which is then fed into a temporal Transformer encoder for spatiotemporal modeling to obtain a spatiotemporal historical feature sequence. Finally, the spatiotemporal historical feature sequence and the contextual operating condition feature sequence are fed into a dual-stream collaborative attention network to generate a fusion diagnostic vector reflecting the current health status. This fusion diagnostic vector is input into a trained fault prediction learning model, outputting a health status label and a quantified confidence score. The final output can be synchronously transmitted to the equipment health dashboard, guiding predictive maintenance decisions for heavy-duty gearboxes and providing timely and accurate diagnostic data for the safe and reliable operation of high-power, heavy-duty gearbox transmission devices under complex load conditions.

[0061] Experimental Analysis: Figure 5The confusion matrix of the proposed model's diagnostic performance on five preset fault categories is presented. The highlighted values ​​on the diagonal are close to 100%, intuitively demonstrating that the proposed method has extremely high diagnostic accuracy and recall in normal conditions and across the four different fault categories. The small values ​​off-diagonally reveal slight confusion between specific categories; for example, the model exhibits a very low misclassification rate between gear faults and bearing faults with similar physical vibration characteristics, which aligns with engineering practice. Overall, this confusion matrix strongly demonstrates the model's ability to accurately classify complex and similar fault modes. Figure 6 The performance of the proposed method's model was compared with two baseline models and two ablation configurations using grouped bar charts, thus verifying the technical advancement and structural rationality of the proposed method. The charts show that the accuracy and F1 score of the proposed method are significantly better than all the comparison models. The comparison with ablation experiments is particularly crucial: removing the dynamic graph module or the collaborative fusion module both resulted in a significant decline in model performance. This directly demonstrates that considering the dynamic effects of high-power, heavy-load environments and performing cross-modal deep fusion are core elements for improving diagnostic accuracy. This chart strongly demonstrates the superiority of the proposed multi-stage, multi-modal fusion architecture compared to traditional methods. Figure 7 The results of noise robustness analysis of our proposed method and the comparative model under different signal-to-noise ratio (SNR) environments are presented. The horizontal axis in the figure represents the SNR; a smaller value indicates a greater intensity of injected noise in the signal. The results clearly show that although the performance of all models decreases with increasing noise, the performance curve of our proposed model is the flattest, maintaining a diagnostic accuracy of over 90% even under harsh conditions of strong noise. In contrast, the performance of the baseline model deteriorates sharply. This comparison highlights the effectiveness of our proposed method, particularly its cross-modal collaborative fusion mechanism, in calibrating and denoising microscopic vibration signals using macroscopic operating condition information, thereby ensuring the diagnostic reliability and stability of the model in high-power, heavy-load real-world operating environments.

[0062] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

[0063] While the specific embodiments of the present invention have been described above, they are not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A fault diagnosis method for high-power, heavy-duty gearbox transmission devices based on multimodal deep learning, characterized in that, Includes the following steps: S1, based on a multimodal sensor array, collects real-time data on gearbox operating status and gearbox working conditions; S2, Based on the physical topology of the gearbox, a dynamic weighted sensor topology graph containing force transmission path weights is constructed. Vibration signal data and noise signal data are transformed into dynamic graph node feature sequences that can describe gear meshing impact characteristics, local damage characteristics and energy distribution laws. The macroscopic operating state of the gearbox and the lubrication system state are transformed into contextual operating condition feature sequences that are synchronized with the node time-frequency characteristics. S3 inputs the dynamic weighted sensor topology map and dynamic graph node feature sequence output from S2 into the spatial correlation perception module, outputting a spatial fusion feature sequence; inputs the spatial fusion feature sequence into the spatiotemporal dynamic evolution module to obtain the spatiotemporal history feature sequence; inputs the spatiotemporal history feature sequence combined with the contextual condition feature sequence obtained from S2 into the dual-stream cross-attention network module to obtain the final high-dimensional fusion diagnostic vector; S4 takes the high-dimensional fusion diagnostic vector obtained in S3 and inputs it into the trained fault prediction learning model, outputting the fault category and confidence level.

2. The fault diagnosis method for a high-power, heavy-duty gearbox transmission device based on multimodal deep learning as described in claim 1, characterized in that: The gearbox operating status data includes vibration signal data, noise signal data, ambient temperature time series data, oil temperature time series data, oil pressure time series data, bearing speed time series data, and bearing torque time series data. The gearbox operating data includes bearing load rating, gearbox input power, gearbox output power, gear meshing frequency, lubricating oil flow rate, and oil pump pressure.

3. The fault diagnosis method for a high-power, heavy-duty gearbox transmission device based on multimodal deep learning as described in claim 2, characterized in that: In step S2, firstly, a dynamic sensor topology graph containing force transmission path weights is constructed based on the physical topology of the gearbox to express the mechanical coupling relationship between sensors. Fluctuations in bearing load level, bearing speed time series data, bearing torque time series data, and ambient temperature time series data will dynamically adjust the edge weights of the graph structure to reflect changes in the power path. Subsequently, vibration signal data and noise signal data are transformed into time-frequency feature sequences that can describe gear meshing impact characteristics, local damage characteristics, and energy distribution patterns through sliding window, short-time Fourier transform, and wavelet packet decomposition methods. At the same time, oil temperature time series data, oil pressure time series data, as well as low-frequency signals of gearbox input power, gearbox output power, gear meshing frequency, lubricating oil flow rate, and oil pump pressure are input into a multilayer perceptron to generate a context feature vector sequence describing the macroscopic health state and lubrication conditions. The final output can reflect the dynamic weighted sensor topology diagram of dynamic coupling, the dynamic graph node feature sequence reflecting local damage of gears / bearings, and the contextual operating condition feature sequence describing the macroscopic operating conditions of oil temperature and oil pressure fluctuations.

4. The fault diagnosis method for a high-power, heavy-duty gearbox transmission device based on multimodal deep learning as described in claim 3, characterized in that: The specific process for obtaining the dynamic weighted sensor topology map is as follows: First, based on the obtained 3D sensor layout diagram, all torque sensors, oil sensors, speed sensors, and temperature sensors are mapped as independent nodes in the graph structure. Second, based on the gearbox structural drawings and dynamic transmission chain, initial static edge connections are established according to gear meshing relationships, bearing supports, and force transmission paths. This is abstracted into an undirected weighted graph with node characteristics, forming the basic topology, formalized as follows: ； The high-frequency vibration sensor, acoustic sensor, oil sensor, speed sensor, and torque sensor are regarded as independent nodes, and the node set V represents the sensor position. Node features It includes real-time sensed multimodal data vectors; torque node features include instantaneous torque and load fluctuation rate; temperature node features include real-time oil temperature and temperature rise rate; pressure node features include absolute oil pressure and oil pressure fluctuation rate; speed node features include speed and speed fluctuation rate; vibration node features include impact characteristic index and fault modulation envelope energy. The static boundary is constructed based on the inherent power transmission and medium transmission relationships determined by the mechanical structure of the gearbox. This includes the power transmission side, which represents the torque and speed propagation path caused by gear meshing, shaft connection, and transmission stage arrangement. The medium transmission relationship is determined by the fluid path. edge weight This represents the coupling strength between two nodes. The node edges adopt the idea of ​​static connection and dynamic weight. The initial value of the static edge weight is a uniform value of 1, which is used to determine the connectivity graph, so that the dynamic weight update module can start learning from the unbiased initial state. Simultaneously, a load-driven dynamic edge weight update mechanism is constructed. Based on vibration signal data, bearing load level, bearing torque time series data, oil temperature time series data, oil pressure time series data, and bearing speed time series data, a multi-source coupling model is built, including a load effect model, a temperature effect model, and an oil pressure effect model, to calculate the adjustment coefficients of each mode.

5. The fault diagnosis method for a high-power, heavy-duty gearbox transmission device based on multimodal deep learning as described in claim 3, characterized in that: The specific process for obtaining the feature sequence of the dynamic graph node is as follows: The input consists of structural vibration signals and acoustic noise signals. First, a sliding window mechanism is used to uniformly divide the long-sequence vibration and noise signals into time segments. By setting a fixed window length and window overlap rate, the continuous time-domain waveform is strictly divided into a series of short time segments aligned on the time axis, ensuring that all sensor signals form corresponding window segments at the same time step. Second, a Short-Time Fourier Transform (STFT) is performed on each time segment to obtain its two-dimensional time-frequency spectrum, reflecting the change of energy distribution at different frequencies over time. Third, each two-dimensional time-frequency spectrum is transformed into a fixed-length one-dimensional feature vector through frequency domain pooling, energy bandwidth aggregation, or convolution feature compression, making it a snapshot of the time-frequency features of the corresponding sensor node at that time step. Finally, a dynamic graph node feature sequence is output, organized as a list sorted by time, with each time step containing a set of time-frequency feature vectors for all sensor nodes.

6. The fault diagnosis method for a high-power, heavy-duty gearbox transmission device based on multimodal deep learning as described in claim 3, characterized in that: The specific process for obtaining the context condition feature sequence is as follows: The inputs include oil temperature time series data, oil pressure time series data, oil flow rate, oil pump pressure, and operating parameters such as input power, output power, and gear meshing frequency. First, to achieve cross-modal time alignment, a sliding window mechanism is used to synchronously segment the original time series of all operating data. The multidimensional operating condition signal is divided into a series of time segments, so that each time segment corresponds to the time segments of all sensor signals in the dynamic graph node feature sequence acquisition on the time axis, achieving time step-level unification. Secondly, within each time segment, statistical context features are extracted through an independent operating condition feature extraction network, including the average oil temperature and rate of change, the average oil pressure and pulsation amplitude, the oil flow stability factor, the average oil pump pressure, the load factor, and the meshing frequency features, as descriptors characterizing macroscopic operating condition changes. The operating condition feature extraction network consists of an input layer, multiple ReLU-activated nonlinear hidden layers, and an output layer, used to capture the nonlinear coupling and multivariate interaction relationships between operating condition parameters, and finally outputs a context operating condition feature sequence. This sequence is strictly synchronized with the dynamic graph node feature sequence on the time step, so that each time point has the microscopic time-frequency characteristics from the sensor and the macroscopic system state from the operating condition.

7. The fault diagnosis method for a high-power, heavy-duty gearbox transmission device based on multimodal deep learning as described in claim 1, characterized in that: The spatial correlation sensing module is specifically: A graph attention (GAT) module is constructed, consisting of three stacked graph attention layers. Each layer is a multi-head graph attention sub-layer with residual connections and layer normalization. Graph attention network inference is performed independently for each time step in the sequence. For any time step, GAT takes the dynamic graph node features at the current time as input and performs neighborhood feature aggregation based on the weighted adjacency matrix at that time. Through the self-attention mechanism, GAT adaptively calculates the attention coefficient based on the correlation of features of adjacent nodes in the topology graph, so that the edge weights not only reflect the device structure dependence. Subsequently, the nodes update their own representations by weighted fusion based on the edge weights, realizing spatial modeling of the device structure response, and finally outputting a spatial fusion feature sequence. Each element in the sequence corresponds to a time step, containing the spatial features of all nodes after GAT aggregation, which is used to describe the overall structural response mode of the gearbox.

8. The fault diagnosis method for a high-power, heavy-duty gearbox transmission device based on multimodal deep learning as described in claim 7, characterized in that: The spatiotemporal dynamic evolution module consists of a multi-layered stacked temporal Transformer encoder. Each encoder block contains a multi-head self-attention mechanism and a feedforward neural network. The multi-head self-attention mechanism is used to calculate the dependency between any two time steps globally, enabling the model to simultaneously capture short-term transient changes, including high-frequency impacts caused by oil film rupture, as well as long-term gradual trends, including the evolution of gear pitting, the long-term accumulation of fatigue spalling in bearing raceways, and the impact of oil film degradation on vibration energy distribution. The feedforward neural network is used to nonlinearly enhance the output of the self-attention mechanism. All sub-layers contain residual connections and layer normalization. Finally, a spatiotemporal history feature sequence is output, in which each time step encodes the spatial aggregation features of the current moment and also incorporates the dynamic evolution information of its preceding moments.

9. The fault diagnosis method for a high-power, heavy-duty gearbox transmission device based on multimodal deep learning as described in claim 8, characterized in that: The dual-stream cross-attention network module is specifically as follows: First, using the contextual operating condition feature sequence as the query term, the attention distribution is calculated for the spatiotemporal historical feature sequence, enabling the macroscopic operating condition to selectively focus on the microscopic structural changes that have the most significant impact. The reverse attention path uses the spatiotemporal historical feature sequence as the query term and performs reverse attention on the contextual operating condition feature sequence, enabling the microscopic structural dynamics to actively extract the key operating condition variables on which they depend. The outputs of the bidirectional cross-attention are passed through their respective feedforward fully connected networks, and the two sequences are updated using residual connections and layer normalization to provide input for the next layer of fusion. As the multi-layer collaborative fusion progresses layer by layer, the information of the two branches is deeply coupled at different levels of abstraction. Finally, at the end of the fusion network, the output vector corresponding to the position of the classification embedding vector is extracted to obtain the final high-dimensional fusion diagnostic vector. This vector comprehensively encodes the topological features of the gearbox structure, the dynamic evolution law across time scales, the vibration and acoustic micro-response modes, operating conditions, oil health and load trends.

10. The fault diagnosis method for a high-power, heavy-duty gearbox transmission device based on multimodal deep learning as described in claim 1, characterized in that: The fault prediction learning model employs a three-layer linear mapping to map high-dimensional fused diagnostic vectors to the fault category space, outputting the fault category probability distribution.

Images