Multi-sensor cooperative detection method and system of fan cabin hanging rail inspection robot

Abstract

The invention belongs to the technical field of fan operation and maintenance, and particularly discloses a fan cabin hanging rail inspection robot multi-sensor cooperative detection method and system, multi-physical field data can be synchronously acquired through a multi-modal sensor array, and the data quality can be improved by performing layered preprocessing and cooperative calibration on the multi-physical field data. Reliable data support is provided for fault diagnosis; a diagnosis model fusing a graph neural network and an attention mechanism is adopted, the fault type under multi-factor coupling can be accurately recognized, and the intelligent level of operation and maintenance of a fan cabin is improved; the method can achieve the automatic recognition of fault types, fault degrees and fault related factors, provides a visual fault diagnosis result for operation and maintenance personnel, facilitates the operation and maintenance personnel to carry out the operation and maintenance work in a targeted manner, reduces the operation and maintenance difficulty and cost, and improves the operation and maintenance efficiency of a fan cabin. The adaptability and the expansibility are high, the operation and maintenance requirements of the fan cabin under different working conditions can be met, and the wide engineering application prospect is achieved.

Description

Technical Field

[0001] This invention belongs to the field of wind turbine operation and maintenance technology, specifically relating to a multi-sensor collaborative detection method and system for a wind turbine nacelle rail inspection robot. Background Technology

[0002] The internal environment of a wind turbine nacelle is complex, containing critical components such as gearboxes, generators, and bearings. Its operational status directly determines the turbine's power generation efficiency and safety stability. As a core piece of equipment for nacelle maintenance, the wind turbine nacelle rail-mounted inspection robot can replace manual labor in high-altitude and high-risk environments for inspection and fault diagnosis, and has become a research hotspot in wind turbine maintenance technology.

[0003] Existing wind turbine nacelle rail-mounted inspection robot maintenance technologies are mostly focused on two main areas: motion control (such as path planning and obstacle avoidance navigation) and single fault detection (such as bolt loosening detection and oil leakage detection). Although these technologies have improved the automation level of nacelle maintenance to some extent, the accuracy of fault diagnosis under complex working conditions is still significantly insufficient, making it difficult to meet actual maintenance needs. The core pain point is the problem of fault misjudgment and missed detection in multi-physics coupling scenarios.

[0004] The interior of a wind turbine nacelle contains multiple physical fields, including temperature, vibration, and electromagnetic interference. These fields are coupled and influence each other, causing the failures of critical components to exhibit multi-factor correlation characteristics. For example, the occurrence and development of bearing wear failures are simultaneously affected by the high-temperature environment inside the nacelle, equipment operating vibration, and electromagnetic interference. Monitoring data from a single physical field cannot fully reflect the nature of the failure.

[0005] Existing fault diagnosis solutions for wind turbine nacelle rail inspection robots mostly rely on data collection from a single type of sensor (such as vibration or temperature sensors) or use single-dimensional diagnostic models for fault identification, failing to fully consider the impact of multi-physics coupling. On the one hand, the data collected by a single sensor has limited dimensions and cannot capture fault characteristics under the interaction of multiple factors. On the other hand, single-dimensional diagnostic models struggle to uncover the correlations between data from different physical fields, easily leading to misdiagnosis and missed faults. For example, traditional vibration analysis methods, under the combined effects of high temperature and electromagnetic interference, easily overlook early characteristics of bearing wear, causing fault expansion and increasing maintenance costs and safety risks.

[0006] Therefore, there is an urgent need for a collaborative detection technology solution for wind turbine nacelle rail-mounted inspection robots that can achieve accurate multi-dimensional data collection and precise identification of faults related to multiple factors, in order to solve the problems of low fault diagnosis accuracy and weak anti-interference ability of existing technologies. Summary of the Invention

[0007] The purpose of this invention is to provide a multi-sensor collaborative detection method and system for a wind turbine nacelle rail inspection robot, in order to solve the above-mentioned problems existing in the prior art.

[0008] To achieve the above objectives, the present invention adopts the following technical solution: Firstly, a multi-sensor collaborative detection method for a wind turbine nacelle rail inspection robot is provided, including: The multidimensional sensor dataset transmitted by the main controller of the wind turbine nacelle rail inspection robot is obtained. The multidimensional sensor dataset is synchronously collected by the main controller of the wind turbine nacelle rail inspection robot from the multimodal sensor array at the inspection end. The multidimensional sensor dataset includes vibration sensing signals, infrared thermal imaging data and electromagnetic sensing signals. Vibration sensing signals, infrared thermal imaging data and electromagnetic sensing signals are preprocessed and feature extracted respectively to obtain vibration feature data, infrared temperature feature data and electromagnetic feature data. Vibration characteristic data, infrared temperature characteristic data, and electromagnetic characteristic data are calibrated together to obtain a multi-physics field calibrated dataset, which includes calibrated vibration characteristic data, infrared temperature characteristic data, and electromagnetic characteristic data. A multi-physics field (MPF) collaborative calibration dataset is input into a pre-trained MPF coupled fault diagnosis model for fault diagnosis, yielding fault diagnosis results. These results include fault type, fault severity, and fault-related factors. The MPF coupled fault diagnosis model comprises an input layer, a feature fusion layer, a coupling diagnosis layer, and an output layer connected sequentially. The input layer receives the MPF collaborative calibration dataset and concatenates the calibrated vibration feature data, infrared temperature feature data, and electromagnetic feature data into a multi-dimensional input vector. The feature fusion layer adaptively weights and fuses the multi-dimensional input vector using an attention mechanism to obtain a coupled feature vector. The coupling diagnosis layer uses a graph neural network to mine the multi-physics coupling relationship of the coupled feature vector and extract fault feature vectors. The output layer maps the fault feature vectors to fault diagnosis results using a fully connected layer and an activation function, and outputs the fault diagnosis results. The fault diagnosis results are transmitted to the main controller of the wind turbine nacelle rail inspection robot.

[0009] In one possible design, the preprocessing and feature extraction of vibration sensing signals, infrared thermal imaging data, and electromagnetic sensing signals to obtain vibration feature data, infrared temperature feature data, and electromagnetic feature data respectively includes: The vibration sensing signal is denoised using a wavelet threshold denoising algorithm to obtain the denoised vibration sensing signal. The denoised vibration sensing signal is then analyzed in the time domain and frequency domain to extract time domain features and frequency domain features. The time domain features and frequency domain features are then normalized and used to form vibration feature data. The infrared thermal imaging data is subjected to grayscale correction and histogram equalization to obtain enhanced infrared thermal imaging data. Temperature features of key parts are extracted from the infrared thermal imaging data to obtain various temperature features. The temperature features are then normalized and used to form infrared temperature feature data. An adaptive filtering algorithm is used to filter the electromagnetic sensing signal to obtain the filtered electromagnetic sensing signal. The amplitude, phase and frequency features are then extracted from the filtered electromagnetic sensing signal to obtain the amplitude, phase and frequency features. The amplitude, phase and frequency features are then normalized and used to form electromagnetic feature data.

[0010] In one possible design, the time-domain features include peak value, RMS value, kurtosis, and waveform factor; the frequency-domain features include characteristic frequency and harmonic amplitude; the temperature features include maximum temperature, minimum temperature, average temperature, and temperature gradient; the amplitude features include peak amplitude and RMS amplitude; the phase features include phase offset; and the frequency features include dominant frequency and harmonic frequency.

[0011] In one possible design, the data co-calibration of vibration characteristic data, infrared temperature characteristic data, and electromagnetic characteristic data to obtain a multi-physics co-calibration dataset includes: The vibration characteristic data is linearly calibrated using preset calibration vibration data to obtain calibrated vibration characteristic data. The infrared temperature characteristic data is linearly calibrated using preset calibration temperature data to obtain calibrated infrared temperature characteristic data. The electromagnetic characteristic data is linearly calibrated using preset calibration electromagnetic data to obtain calibrated electromagnetic characteristic data. The calibrated vibration characteristic data, infrared temperature characteristic data, and electromagnetic characteristic data are saved in time correlation to form a multi-physics collaborative calibration dataset.

[0012] In one possible design, the input layer of the multiphysics coupled fault diagnosis model has an input vector dimension of 22. The feature fusion layer adopts a multi-head self-attention mechanism with 6 heads and a linear transformation dimension of 22. It calculates the internal correlation weights of features through self-attention and the correlation weights between different features through cross-attention, outputting a 22-dimensional coupled feature vector. The coupled diagnosis layer adopts a graph neural network with two graph convolutional layers. The output dimension of the first graph convolutional layer is 32, and the output dimension of the second graph convolutional layer is 16. The graph pooling layer of the graph neural network adopts global average pooling, outputting a 16-dimensional fault feature vector. The fully connected layer of the output layer consists of two layers. The first fully connected layer has 32 neurons and uses the ReLU activation function, while the second fully connected layer has 8 neurons and uses the Softmax activation function.

[0013] In one possible design, before inputting the multiphysics co-calibration dataset into a pre-trained multiphysics coupled fault diagnosis model for fault diagnosis, the method further includes: A multi-physics field coupled fault diagnosis model is constructed, and the model is trained using several sets of positive samples and several sets of negative samples from a pre-set multi-physics field co-calibration dataset to obtain the trained multi-physics field coupled fault diagnosis model. The positive samples of the multi-physics field co-calibration dataset are labeled with a normal label, and the negative samples are labeled with fault type, fault severity, and fault-related factors.

[0014] Secondly, a multi-sensor collaborative detection system for a wind turbine nacelle rail inspection robot is provided, comprising a multi-modal sensor array, a main controller, and an edge computing module. The multi-modal sensor array is installed at the inspection end of the wind turbine nacelle rail inspection robot and includes a vibration sensor, an infrared thermal imaging sensor, and an electromagnetic sensor. The vibration sensor is used to collect vibration sensing signals of the component under test, the infrared thermal imaging sensor is used to collect infrared thermal imaging data of the component under test, and the electromagnetic sensor is used to collect electromagnetic sensing signals of the component under test. The main controller and the edge computing module are installed inside the wind turbine nacelle rail inspection robot. The main controller is used to perform timing synchronization control of the vibration sensor, the infrared thermal imaging sensor, and the electromagnetic sensor, and synchronously collects the vibration sensing signals of the vibration sensor, the infrared thermal imaging data of the infrared thermal imaging sensor, and the electromagnetic sensing signals of the electromagnetic sensor to form a multi-dimensional sensing dataset. The multi-dimensional sensing dataset is transmitted to the edge computing module in real time. The edge computing module is used to execute any one of the multi-sensor collaborative detection methods for the wind turbine nacelle rail inspection robot described in the first aspect.

[0015] In one possible design, the main controller uses a microcontroller or DSP chip, and the main controller synchronizes the clock signal to the vibration sensor, infrared thermal imaging sensor and electromagnetic sensor via CAN bus, Ethernet PTP synchronization protocol or UART bus.

[0016] In one possible design, the vibration sensor is a piezoelectric vibration sensor or a capacitive vibration sensor, the infrared thermal imaging sensor is a non-contact infrared thermal imaging sensor or a cooled infrared thermal imaging sensor, and the electromagnetic sensor is a Hall effect electromagnetic sensor or an inductive electromagnetic sensor.

[0017] Thirdly, a multi-sensor collaborative detection system for a wind turbine nacelle rail inspection robot is provided, including: Memory, used to store instructions; The processor is configured to read instructions stored in the memory and execute, according to the instructions, any one of the multi-sensor collaborative detection methods for wind turbine nacelle rail inspection robots described in the first aspect above.

[0018] Fourthly, a computer-readable storage medium is provided, on which instructions are stored, which, when executed on a computer, cause the computer to perform any one of the multi-sensor collaborative detection methods for wind turbine nacelle rail inspection robots described in the first aspect. Simultaneously, a computer program product is also provided, which, when executed on a computer, performs any one of the multi-sensor collaborative detection methods for wind turbine nacelle rail inspection robots described in the first aspect.

[0019] Beneficial Effects: This invention can simultaneously collect multi-physical field data through a multi-modal sensor array. By performing hierarchical preprocessing and collaborative calibration on the multi-physical field data, data quality can be improved, providing reliable data support for fault diagnosis and avoiding misjudgment and missed detection caused by insufficient data dimensions from a single sensor. The invention employs a diagnostic model that integrates graph neural networks and attention mechanisms for fault diagnosis. This model can adaptively allocate feature weights through the attention mechanism to highlight key fault features, and it can also mine the intrinsic correlations between multi-physical field features through the graph neural network, accurately identifying fault types under multi-factor coupling and improving the intelligence level of wind turbine nacelle operation and maintenance. It can automatically identify fault types, fault severity, and fault-related factors, providing operation and maintenance personnel with intuitive fault diagnosis results, facilitating targeted operation and maintenance work, reducing operation and maintenance difficulty and costs, and improving the efficiency of wind turbine nacelle operation and maintenance. Furthermore, it has strong adaptability and scalability, capable of adapting to the operation and maintenance needs of wind turbine nacelles under different operating conditions, and has broad engineering application prospects. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a flowchart illustrating the method in Embodiment 1 of the present invention; Figure 2 This is a schematic diagram of the system configuration in Embodiment 2 of the present invention; Figure 3 This is a schematic diagram of the system configuration in Embodiment 3 of the present invention. Detailed Implementation

[0022] It should be noted that the descriptions of these embodiments are intended to aid in understanding the invention and do not constitute a limitation thereof. The specific structural and functional details disclosed herein are merely for describing exemplary embodiments of the invention. However, the invention may be embodied in many alternative forms and should not be construed as being limited to the embodiments described herein.

[0023] It should be understood that, unless otherwise explicitly specified and limited, the corresponding terms should be interpreted broadly. For example, "connection" can be a fixed connection, a detachable connection, or an integral connection; it can be a direct connection or an indirect connection through an intermediate medium; it can be a connection within two components. Those skilled in the art can understand the specific meaning of the above terms in the embodiments according to the specific circumstances.

[0024] Specific details are provided in the following description to provide a complete understanding of the exemplary embodiments. However, those skilled in the art will understand that the exemplary embodiments can be implemented without these specific details. For example, apparatus may be shown in block diagrams to avoid obscuring the examples with unnecessary details. In other embodiments, well-known processes, structures, and techniques may be omitted with non-essential details to avoid obscuring the embodiments.

[0025] Example 1: This embodiment provides a multi-sensor collaborative detection method for a wind turbine nacelle rail inspection robot, which can be applied to corresponding edge computing modules, such as... Figure 1 As shown, the method includes the following steps: S1. Obtain the multidimensional sensor dataset transmitted by the main controller of the wind turbine nacelle rail inspection robot. The multidimensional sensor dataset is obtained by the main controller of the wind turbine nacelle rail inspection robot synchronously collecting data from the multimodal sensor array at the inspection end. The multidimensional sensor dataset includes vibration sensing signals, infrared thermal imaging data, and electromagnetic sensing signals.

[0026] In practical implementation, a multimodal sensor array is pre-deployed at the inspection end of the wind turbine nacelle rail inspection robot. This array includes vibration sensors, infrared thermal imaging sensors, and electromagnetic sensors. The vibration sensors collect vibration signals from the components under test, the infrared thermal imaging sensors collect infrared thermal imaging data, and the electromagnetic sensors collect electromagnetic signals. When the wind turbine nacelle rail inspection robot is operating, its main controller can perform timing synchronization control of the multimodal sensor array. Based on the robot's path planning information, the main controller employs a dual synchronization mechanism of "global clock synchronization + local trigger synchronization" to achieve precise alignment of the multi-sensor acquisition timing. Global clock synchronization: Using the robot's main controller clock (which can be an STM32H743 microcontroller) as a reference, the clock signal is synchronized to each sensor in the multimodal sensor array via the CAN bus, ensuring that the clock error of each sensor is controlled within ±1ms. Alternatively, an Ethernet / PTP synchronization protocol can be used, with the clock synchronization error controlled within ±0.1ms, suitable for high-speed multi-sensor acquisition scenarios (the main controller can be a DSP TMS320F28335 instead of the STM32H743).

[0027] Local trigger synchronization: When the robot moves to the preset inspection point (the position corresponding to the inspected part), the main controller sends a trigger signal to simultaneously start the vibration sensor, infrared thermal imaging sensor, and electromagnetic sensor to collect data. After the data collection is completed, the main controller sends a stop signal, and each sensor stops collecting data synchronously, avoiding timing misalignment of data collected by different sensors and ensuring the temporal correlation of multi-physics data.

[0028] The main controller synchronously collects vibration sensing signals from vibration sensors, infrared thermal imaging data from infrared thermal imaging sensors, and electromagnetic sensing signals from electromagnetic sensors to form a multi-dimensional sensing dataset, which is then transmitted to the edge computing module in real time.

[0029] S2. Preprocess and extract features from the vibration sensing signal, infrared thermal imaging data, and electromagnetic sensing signal respectively to obtain vibration feature data, infrared temperature feature data, and electromagnetic feature data.

[0030] In practical implementation, the edge computing module can perform hierarchical preprocessing on the multidimensional sensor dataset to remove noise interference, unify the data format, and provide high-quality data for subsequent fault diagnosis models. Specifically, this includes: A wavelet thresholding denoising algorithm is used to denoise the vibration sensing signal (e.g., using a db4 wavelet basis, setting the decomposition level to 5 levels, and setting the wavelet threshold to 0.05) to remove interference from ambient noise in the cabin and the sensor's own noise, resulting in a denoised vibration sensing signal. Then, time-domain and frequency-domain analyses are performed on the denoised vibration sensing signal to extract time-domain features (including peak value, RMS value, kurtosis, and waveform factor) and frequency-domain features (including characteristic frequencies and harmonic amplitudes). These time-domain and frequency-domain features are then normalized (using the min-max normalization method), and the normalized time-domain and frequency-domain features are used to construct vibration feature data. Alternatively, Empirical Mode Decomposition (EMD) can be used instead of wavelet thresholding denoising, which is more suitable for denoising nonlinear and non-stationary vibration signals. The extracted frequency-domain features can increase power spectral density and spectral kurtosis, improving fault feature identification.

[0031] The infrared thermal imaging data undergoes grayscale correction and histogram equalization to obtain enhanced infrared thermal imaging data. Temperature features of key areas are extracted from the infrared thermal imaging data to obtain various temperature features (including maximum temperature, minimum temperature, average temperature, and temperature gradient). These temperature features are then normalized, and the normalized temperature features are used to construct infrared temperature feature data. Alternatively, adaptive histogram equalization (CLAHE) can be used instead of traditional histogram equalization to avoid image distortion due to excessive brightness. The temperature features can also increase temperature variance and hotspot area ratio, making it suitable for detecting localized overheating faults.

[0032] An adaptive filtering algorithm (filtering order set to 8) is used to filter the electromagnetic sensing signal, removing 50Hz power frequency interference and noise, resulting in a filtered electromagnetic sensing signal. Then, amplitude, phase, and frequency features are extracted from the filtered signal, yielding amplitude features (including peak and effective amplitude), phase features (including phase shift), and frequency features (including the dominant frequency and harmonic frequencies). These features are then normalized (to ensure dimensionality consistency with vibration and temperature feature data), and the normalized amplitude, phase, and frequency features are used to construct electromagnetic feature data. Alternatively, wavelet packet decomposition can be used instead of adaptive filtering to more accurately remove multi-frequency noise interference. The extracted electromagnetic features can increase harmonic distortion and signal-to-noise ratio (SNR), improving the accuracy of electromagnetic interference anomaly identification.

[0033] S3. Perform data co-calibration on vibration characteristic data, infrared temperature characteristic data and electromagnetic characteristic data to obtain a multi-physics field co-calibration dataset, wherein the multi-physics field co-calibration dataset includes calibrated vibration characteristic data, infrared temperature characteristic data and electromagnetic characteristic data.

[0034] In practical implementation, the edge computing module can use a pre-set calibration dataset (including calibration vibration data, calibration temperature data, and calibration electromagnetic data) to perform linear calibration on the vibration characteristic data, infrared temperature characteristic data, and electromagnetic characteristic data, obtaining calibrated vibration characteristic data, infrared temperature characteristic data, and electromagnetic characteristic data. Then, the calibrated vibration characteristic data, infrared temperature characteristic data, and electromagnetic characteristic data are saved with time correlation (the data of each dimension after calibration can be saved according to timestamps), forming a multi-physics collaborative calibration dataset. Alternatively, a neural network calibration algorithm can be used to replace the linear calibration method to adapt to multi-sensor nonlinear error correction.

[0035] S4. Input the multi-physics field collaborative calibration dataset into the pre-trained multi-physics field coupled fault diagnosis model for fault diagnosis to obtain fault diagnosis results. The fault diagnosis results include fault type, fault severity, and fault correlation factors. The multi-physics field coupled fault diagnosis model includes an input layer, a feature fusion layer, a coupling diagnosis layer, and an output layer connected in sequence. The input layer is used to receive the multi-physics field collaborative calibration dataset and concatenate the calibrated vibration feature data, infrared temperature feature data, and electromagnetic feature data into a multi-dimensional input vector. The feature fusion layer is used to adaptively weight and fuse the multi-dimensional input vector through an attention mechanism to obtain a coupled feature vector. The coupling diagnosis layer is used to mine the multi-physics field coupling relationship of the coupled feature vector through a graph neural network to extract the fault feature vector. The output layer is used to map the fault feature vector into a fault diagnosis result through a fully connected layer and an activation function, and output the fault diagnosis result.

[0036] In practical implementation, the edge computing module can input the multi-physics collaborative calibration dataset into a pre-trained multi-physics coupled fault diagnosis model for fault diagnosis, obtaining fault diagnosis results. These results include fault type, fault severity, and fault-related factors. The multi-physics coupled fault diagnosis model comprises an input layer, a feature fusion layer, a coupled diagnosis layer, and an output layer connected sequentially, wherein: Input layer: This layer receives the multi-physics collaborative calibration dataset and uses feature concatenation to combine the calibrated vibration, infrared temperature, and electromagnetic features into a multi-dimensional input vector. The input dimension of the multi-dimensional input vector is dynamically adjusted based on the actual number of features collected, preferably 32-64 dimensions. It can also be adjusted to 16-32 dimensions (suitable for scenarios with fewer features) or 64-128 dimensions (suitable for scenarios with more features and more complex fault types). The multi-dimensional feature integration method can use feature concatenation + principal component analysis (PCA) for dimensionality reduction, reducing redundant features and improving model computation speed.

[0037] Feature Fusion Layer: An attention mechanism is used to construct feature fusion units to mine the correlation weights between features of different physical fields, achieving adaptive fusion of multimodal features. Specifically, this includes: linearly transforming the vibration, temperature, and electromagnetic feature data from the input layer to map them to the same feature space; calculating the correlation weights within each type of feature data through a self-attention mechanism, and simultaneously calculating the correlation weights between different types of feature data (e.g., the correlation weights between vibration and temperature features, and electromagnetic and temperature features) through a cross-attention mechanism; and weighting and fusing the three types of feature data based on the calculated weight coefficients, outputting a fused coupled feature vector that highlights features highly contributing to fault diagnosis and suppresses interference from irrelevant features. Alternatively, a multi-head attention mechanism (2-4 heads to suit lightweight requirements) or a gated attention mechanism can be used to replace the existing multi-head self-attention + cross-attention combination, reducing model parameters and improving inference speed. Feature fusion can employ a hybrid approach of concatenation fusion and attention weighting to further enhance the feature fusion effect.

[0038] Coupled Diagnostic Layer: A graph neural network (GNN) is used to construct coupled diagnostic units to identify causal relationships between multi-physics factors, achieving accurate diagnosis of coupled faults. Specifically, this includes: 1. Constructing a Graph Structure: Using each feature in the fused coupled feature vector as a node and the association weights between different features as edges, a multi-physics feature association graph is constructed. Nodes in the feature association graph represent single physical field features (such as vibration peak value, maximum temperature, electromagnetic amplitude), and edges represent the coupling relationships between features (positive correlation, negative correlation, causal relationship). 2. Graph Neural Network Propagation: The feature association graph is iteratively propagated through graph convolutional layers to capture coupling association information between different nodes (features), automatically mining causal relationships between multi-physics factors (such as the causal relationship between abnormal electromagnetic interference amplitude and bearing temperature rise, and the causal relationship between abnormal vibration kurtosis and bearing wear). Alternatively, Graph Convolutional Networks (GCN) and Graph Sampling and Aggregation Networks (GraphSAGE) can be used instead of Graph Attention Networks (GAT). GCN is suitable for lightweight scenarios, while GraphSAGE is suitable for large-scale feature map scenarios. The number of graph convolutional layers can be adjusted to 1-2 layers (lightweight) or 3-4 layers (high accuracy requirements). 3. Fault Feature Extraction: The propagated features are reduced in dimensionality using graph pooling layers to extract the core features that can characterize the coupled faults, and output a fault feature vector for subsequent fault classification. Alternatively, Top-K pooling or attention pooling can be used instead of global average pooling for graph pooling layers. Top-K pooling can highlight key fault features, while attention pooling can adaptively allocate pooling weights, improving the accuracy of fault feature extraction.

[0039] Output layer: A fully connected layer combined with the Softmax activation function is used to map the fault feature vector output by the coupled diagnostic layer to fault diagnosis results. These results include fault type (e.g., bearing wear, gearbox failure, abnormal electromagnetic interference, abnormal temperature), fault severity (mild, moderate, severe), and related factors (e.g., "bearing wear, related factors: high temperature + electromagnetic interference"). Alternatively, the Sigmoid activation function can be used (suitable for binary classification fault scenarios). The fully connected layer can use a single hidden layer structure (16-32 neurons, lightweight) to replace the existing double hidden layer structure, reducing model computation. The fault diagnosis results can increase the probability of fault occurrence, enhancing the operational and maintenance reference value.

[0040] For example, when constructing a multiphysics coupled fault diagnosis model, the model structure parameters can be set as follows: Input layer: The input vector has a dimension of 22, and the three types of feature data are integrated by feature concatenation. Feature fusion layer: adopts a multi-head self-attention mechanism with 6 heads and a linear transformation dimension of 22. It calculates the correlation weights within features through self-attention and the correlation weights between different features through cross-attention, outputting a 22-dimensional coupled feature vector. Coupled diagnostic layer: A graph neural network is used, with two graph convolutional layers. The output dimension of the first graph convolutional layer is 32-dimensional, and the output dimension of the second graph convolutional layer is 16-dimensional. The graph pooling layer of the graph neural network adopts global average pooling and outputs a 16-dimensional fault feature vector. Output layer: The fully connected layer consists of two layers. The first fully connected layer has 32 neurons and uses the ReLU activation function. The second fully connected layer has 8 neurons (corresponding to 4 fault types + 3 severity levels + 1 normal state) and uses the Softmax activation function to output the diagnostic results.

[0041] In the training and inference phases of the multiphysics coupled fault diagnosis model: 1. Dataset Construction: Using the same method as the multi-sensor collaborative acquisition described above, multi-physics field data is collected under normal operating conditions and different coupled fault conditions of the wind turbine nacelle (such as bearing wear + high temperature, bearing wear + electromagnetic interference, gearbox fault + vibration + abnormal temperature, etc.). Training, validation, and testing datasets are constructed (70% training, 15% validation, 15% testing, or 60% training, 20% validation, and 20% testing). Positive samples in the multi-physics field collaborative calibration dataset are labeled with normal labels, while negative samples are labeled with fault type, fault severity, and fault-related factors. Data augmentation techniques (such as signal shifting, noise superposition, and feature perturbation) can be used to expand the dataset, improve model generalization ability, and add fault location labels to the annotations, further enhancing the targeted nature of operation and maintenance.

[0042] 2. Model Initialization: Set the hyperparameters of the multiphysics coupled fault diagnosis model, including the number of attention heads (preferably 4-8 heads), the number of graph convolutional layers in the graph neural network (preferably 2-3 layers), the number of neurons in the fully connected layers, the learning rate (preferably 0.001-0.01), and the number of iterations (preferably 100-200 epochs). Initialize the weight parameters of each layer of the model using a random normal distribution initialization method. Alternatively, the hyperparameters can be adjusted to: 2-4 attention heads, 1-4 graph convolutional layers, a learning rate of 0.0001-0.05, and 50-300 iterations, to adapt to different accuracy and speed requirements. Xavier initialization or He initialization can be used instead of random normal distribution initialization for weight initialization to improve the model convergence speed.

[0043] 3. Model Training: Input the training dataset into the multiphysics coupled fault diagnosis model, using the cross-entropy loss function as the training loss function to calculate the error between the model's output fault diagnosis results and the labeled values. Use the Adam optimizer to backpropagate and update the model parameters, minimizing the training loss. After each training epoch, validate the model using the validation dataset, adjusting hyperparameters (such as learning rate decay and regularization parameters) to avoid overfitting. Stop training and save the optimal model parameters when the validation set accuracy stabilizes (fluctuation less than 1%) and does not improve for 5 consecutive epochs. Alternatively, validate every 10 epochs during training. Stop training and save the optimal model parameters when the validation set accuracy reaches 98% or higher and does not improve for 5 consecutive epochs. The model accuracy after training is 98.5%, with a false positive rate ≤1.2% and a false negative rate ≤0.8%. For loss functions, focal loss (suitable for imbalanced sample scenarios) and mean squared error loss (suitable for regression-based fault diagnosis) can be used instead of cross-entropy loss function; for optimizers, SGD optimizer (suitable for lightweight scenarios) and RMSprop optimizer can be used instead of Adam optimizer; for regularization, L1 regularization and Dropout (dropout rate 0.1-0.3) can be used instead of weight decay to improve the model's generalization ability and avoid overfitting.

[0044] 4. Fault Reasoning: Real-time multi-physics feature data, collected and preprocessed by multiple sensors, is input into a trained multi-physics coupled fault diagnosis model. The model outputs fault diagnosis results through feature fusion and graph neural network coupling analysis. Simultaneously, it outputs the correlation weights of each physical field factor, clarifying the main causes and coupling relationships of the fault, providing precise basis for wind turbine nacelle operation and maintenance. For example, when abnormal vibration kurtosis, increased bearing surface temperature, and abnormal electromagnetic interference amplitude are collected, the model outputs "Bearing wear (moderate), correlated factors: high temperature + electromagnetic interference," along with the correlation weights of the three factors (temperature 0.45, electromagnetic interference 0.35, vibration 0.20). Maintenance personnel can use this result to specifically check the bearing condition and perform lubrication, cooling, or electromagnetic shielding treatments, achieving precise operation and maintenance. Alternatively, fault reasoning can adopt a batch reasoning method (batch size 8-32 groups) to improve real-time processing efficiency; the correlation weights can be output as normalized values ​​(0-1 range) and labeled with the correlation strength level (strong, medium, weak), facilitating maintenance personnel to quickly understand the fault correlation relationships.

[0045] S5. Transmit the fault diagnosis results to the main controller of the wind turbine nacelle rail inspection robot.

[0046] In practice, the edge computing module inputs the multi-physics field collaborative calibration dataset into the multi-physics field coupled fault diagnosis model for fault diagnosis. After obtaining the fault diagnosis results, the fault diagnosis results can be transmitted to the main controller of the wind turbine nacelle rail inspection robot so that the main controller can perform subsequent processing based on the fault diagnosis results.

[0047] This method can simultaneously acquire multi-physics field data through a multi-modal sensor array. By performing hierarchical preprocessing and collaborative calibration on the multi-physics field data, data quality can be improved, providing reliable data support for fault diagnosis and avoiding misjudgment and missed detection caused by insufficient data dimensions from a single sensor. A diagnostic model integrating graph neural networks and attention mechanisms is used for fault diagnosis. The attention mechanism adaptively allocates feature weights to highlight key fault features, while the graph neural network mines the inherent correlations between multi-physics field features, accurately identifying fault types under multi-factor coupling and improving the intelligence level of wind turbine nacelle operation and maintenance. It can automatically identify fault types, fault severity, and fault-related factors, providing operation and maintenance personnel with intuitive fault diagnosis results, facilitating targeted operation and maintenance work, reducing operation and maintenance difficulty and costs, and improving the efficiency of wind turbine nacelle operation and maintenance. Furthermore, it has strong adaptability and scalability, adapting to the operation and maintenance needs of wind turbine nacelles under different operating conditions, and has broad engineering application prospects.

[0048] Example 2: This embodiment provides a multi-sensor collaborative detection system for a wind turbine nacelle rail inspection robot, such as... Figure 2 As shown, the system includes a multimodal sensor array, a main controller, and an edge computing module. The multimodal sensor array is installed at the inspection end of the wind turbine nacelle rail inspection robot and includes vibration sensors, infrared thermal imaging sensors, and electromagnetic sensors. The vibration sensors are used to collect vibration sensing signals of the tested components, the infrared thermal imaging sensors are used to collect infrared thermal imaging data of the tested components, and the electromagnetic sensors are used to collect electromagnetic sensing signals of the tested components. The main controller and the edge computing module are installed inside the wind turbine nacelle rail inspection robot. The main controller is used to perform timing synchronization control of the vibration sensors, infrared thermal imaging sensors, and electromagnetic sensors, and synchronously collects the vibration sensing signals of the vibration sensors, the infrared thermal imaging data of the infrared sensors, and the electromagnetic sensing signals of the electromagnetic sensors to form a multidimensional sensing dataset. The multidimensional sensing dataset is transmitted to the edge computing module in real time. The edge computing module is used to execute the multi-sensor collaborative detection method of the wind turbine nacelle rail inspection robot in Embodiment 1.

[0049] The vibration sensors are piezoelectric vibration sensors (model PCB 352C65, sensitivity 100mV / g, sampling frequency set to 1500Hz, a total of 4, installed on the front, back and sides of the bottom of the robot inspection end, with the detection direction aligned with the vibration sensitive points of the bearing housing and gearbox output shaft, and fixed by magnetic attraction for easy installation and disassembly) or capacitive vibration sensors (model IEPE 4507B, a total of 2, fixed by bolts).

[0050] The infrared thermal imaging sensor is either a non-contact infrared thermal imaging sensor (model FLIR A655sc, temperature range -20℃~150℃, temperature accuracy ±2℃, resolution 640×480, mounted on a bracket at the front end of the robot inspection end, with the lens vertically aligned with the surface of the gearbox, bearings, and control cabinet; the bracket is adjustable to ensure coverage of all critical components) or a cooled infrared thermal imaging sensor (FLIR C2 compact sensor, temperature range -10℃~150℃, resolution 320×240).

[0051] The electromagnetic sensors used are Hall effect electromagnetic sensors (model ACS712, measurement range ±5A, sampling frequency set to 800Hz, two in total, installed on the side of the robot inspection end near the generator and control cabinet, with the detection direction parallel to the electromagnetic signal propagation direction, and the distance from the component surface controlled at 5-10cm to avoid sensor damage caused by installation too close) or inductive electromagnetic sensors (model LEM HAH100-S, one in total, simplifying the number of sensors and reducing costs).

[0052] The main controller uses an STM32 series microcontroller or DSP chip. It synchronizes clock signals to the vibration sensor, infrared thermal imaging sensor, and electromagnetic sensor via CAN bus, Ethernet PTP synchronization protocol, or UART bus, ensuring that the clock error of each sensor is ≤1ms. The main controller uses the ROS navigation framework for robot path planning, with 10 preset inspection points (corresponding to key components such as bearings, gearboxes, and control cabinets). When the robot reaches a preset inspection point, the main controller sends a high-level trigger signal via GPIO, simultaneously initiating data acquisition by the vibration sensor, infrared thermal imaging sensor, and electromagnetic sensor. The acquisition time for each point is set to 5 seconds. After acquisition is complete, the main controller sends a low-level stop signal, and all sensors synchronously stop acquiring data. The acquired data is transmitted to the main controller in real time.

[0053] Example 3: This embodiment provides a multi-sensor collaborative detection system for a wind turbine nacelle rail inspection robot, such as... Figure 3 As shown, at the hardware level, it includes: The data interface is used to establish data communication between the processor and the main controller; Memory, used to store instructions; The processor is used to read the instructions stored in the memory and execute the multi-sensor collaborative detection method of the wind turbine nacelle rail inspection robot in Embodiment 1 according to the instructions.

[0054] Optionally, the system also includes an internal bus, through which the processor, memory, and data interface can be interconnected. This internal bus can be a PCIe (Peripheral Component Interconnect Eexpress) bus, which can be divided into an address bus, a data bus, a control bus, etc. The memory can include, but is not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Flash Memory, First Input First Output (FIFO), and / or First In Last Out (FILO). The processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0055] Example 4: This embodiment provides a computer-readable storage medium storing instructions. When these instructions are executed on a computer, the computer performs the multi-sensor collaborative detection method for the wind turbine nacelle rail inspection robot described in Embodiment 1. The computer-readable storage medium refers to a data storage medium, which may include, but is not limited to, floppy disks, optical disks, hard disks, flash memory, USB flash drives, and / or Memory Sticks. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.

[0056] This embodiment also provides a computer program product that, when run on a computer, executes the multi-sensor collaborative detection method for the wind turbine nacelle rail inspection robot in Embodiment 1. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.

[0057] Finally, it should be noted that the above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A multi-sensor collaborative detection method for a wind turbine nacelle rail inspection robot, characterized in that, include: The multidimensional sensor dataset transmitted by the main controller of the wind turbine nacelle rail inspection robot is obtained. The multidimensional sensor dataset is synchronously collected by the main controller of the wind turbine nacelle rail inspection robot from the multimodal sensor array at the inspection end. The multidimensional sensor dataset includes vibration sensing signals, infrared thermal imaging data and electromagnetic sensing signals. Vibration sensing signals, infrared thermal imaging data and electromagnetic sensing signals are preprocessed and feature extracted respectively to obtain vibration feature data, infrared temperature feature data and electromagnetic feature data. Vibration characteristic data, infrared temperature characteristic data, and electromagnetic characteristic data are calibrated together to obtain a multi-physics field calibrated dataset, which includes calibrated vibration characteristic data, infrared temperature characteristic data, and electromagnetic characteristic data. A multi-physics field (MPF) collaborative calibration dataset is input into a pre-trained MPF coupled fault diagnosis model for fault diagnosis, yielding fault diagnosis results. These results include fault type, fault severity, and fault-related factors. The MPF coupled fault diagnosis model comprises an input layer, a feature fusion layer, a coupling diagnosis layer, and an output layer connected sequentially. The input layer receives the MPF collaborative calibration dataset and concatenates the calibrated vibration feature data, infrared temperature feature data, and electromagnetic feature data into a multi-dimensional input vector. The feature fusion layer adaptively weights and fuses the multi-dimensional input vector using an attention mechanism to obtain a coupled feature vector. The coupling diagnosis layer uses a graph neural network to mine the multi-physics coupling relationship of the coupled feature vector and extract fault feature vectors. The output layer maps the fault feature vectors to fault diagnosis results using a fully connected layer and an activation function, and outputs the fault diagnosis results. The fault diagnosis results are transmitted to the main controller of the wind turbine nacelle rail inspection robot.

2. The multi-sensor collaborative detection method for a wind turbine nacelle rail-mounted inspection robot according to claim 1, characterized in that, The preprocessing and feature extraction of vibration sensing signals, infrared thermal imaging data, and electromagnetic sensing signals respectively yield vibration feature data, infrared temperature feature data, and electromagnetic feature data, including: The vibration sensing signal is denoised using a wavelet threshold denoising algorithm to obtain the denoised vibration sensing signal. The denoised vibration sensing signal is then analyzed in the time domain and frequency domain to extract time domain features and frequency domain features. The time domain features and frequency domain features are then normalized and used to form vibration feature data. The infrared thermal imaging data is subjected to grayscale correction and histogram equalization to obtain enhanced infrared thermal imaging data. Temperature features of key parts are extracted from the infrared thermal imaging data to obtain various temperature features. The temperature features are then normalized and used to form infrared temperature feature data. An adaptive filtering algorithm is used to filter the electromagnetic sensing signal to obtain the filtered electromagnetic sensing signal. The amplitude, phase and frequency features are then extracted from the filtered electromagnetic sensing signal to obtain the amplitude, phase and frequency features. The amplitude, phase and frequency features are then normalized and used to form electromagnetic feature data.

3. The multi-sensor collaborative detection method for a wind turbine nacelle rail-mounted inspection robot according to claim 2, characterized in that, The time-domain features include peak value, RMS value, kurtosis, and waveform factor; the frequency-domain features include characteristic frequency and harmonic amplitude; the temperature features include maximum temperature, minimum temperature, average temperature, and temperature gradient; the amplitude features include peak amplitude and RMS amplitude; the phase features include phase offset; and the frequency features include dominant frequency and harmonic frequency.

4. The multi-sensor collaborative detection method for a wind turbine nacelle rail-mounted inspection robot according to claim 1, characterized in that, The process of performing collaborative calibration on vibration characteristic data, infrared temperature characteristic data, and electromagnetic characteristic data yields a multi-physics collaborative calibration dataset, including: The vibration characteristic data is linearly calibrated using preset calibration vibration data to obtain calibrated vibration characteristic data. The infrared temperature characteristic data is linearly calibrated using preset calibration temperature data to obtain calibrated infrared temperature characteristic data. The electromagnetic characteristic data is linearly calibrated using preset calibration electromagnetic data to obtain calibrated electromagnetic characteristic data. The calibrated vibration characteristic data, infrared temperature characteristic data, and electromagnetic characteristic data are saved in time correlation to form a multi-physics collaborative calibration dataset.

5. The multi-sensor collaborative detection method for a wind turbine nacelle rail-mounted inspection robot according to claim 1, characterized in that, The multiphysics coupled fault diagnosis model has an input layer with a 22-dimensional input vector. The feature fusion layer employs a multi-head self-attention mechanism with 6 heads and a linear transformation dimension of 22. It calculates the internal correlation weights of features through self-attention and the correlation weights between different features through cross-attention, outputting a 22-dimensional coupled feature vector. The coupled diagnosis layer uses a graph neural network with two graph convolutional layers. The first graph convolutional layer outputs a 32-dimensional vector, and the second graph convolutional layer outputs a 16-dimensional vector. The graph pooling layer uses global average pooling, outputting a 16-dimensional fault feature vector. The output layer consists of two fully connected layers. The first fully connected layer has 32 neurons and uses the ReLU activation function, while the second fully connected layer has 8 neurons and uses the Softmax activation function.

6. The multi-sensor collaborative detection method for a wind turbine nacelle rail-mounted inspection robot according to claim 5, characterized in that, Before inputting the multiphysics co-calibration dataset into the pre-trained multiphysics coupled fault diagnosis model for fault diagnosis, the method further includes: A multi-physics field coupled fault diagnosis model is constructed, and the model is trained using several sets of positive samples and several sets of negative samples from a pre-set multi-physics field co-calibration dataset to obtain the trained multi-physics field coupled fault diagnosis model. The positive samples of the multi-physics field co-calibration dataset are labeled with a normal label, and the negative samples are labeled with fault type, fault severity, and fault-related factors.

7. A multi-sensor collaborative detection system for a wind turbine nacelle rail inspection robot, characterized in that, The system includes a multimodal sensor array, a main controller, and an edge computing module. The multimodal sensor array is installed at the inspection end of a wind turbine nacelle rail inspection robot and includes vibration sensors, infrared thermal imaging sensors, and electromagnetic sensors. The vibration sensors are used to collect vibration sensing signals from the tested components. The infrared thermal imaging sensors are used to collect infrared thermal imaging data from the tested components. The electromagnetic sensors are used to collect electromagnetic sensing signals from the tested components. The main controller and the edge computing module are installed inside the wind turbine nacelle rail inspection robot. The main controller is used to perform timing synchronization control of the vibration sensors, infrared thermal imaging sensors, and electromagnetic sensors, and synchronously collects the vibration sensing signals from the vibration sensors, the infrared thermal imaging data from the infrared thermal imaging sensors, and the electromagnetic sensing signals from the electromagnetic sensors to form a multidimensional sensing dataset. The multidimensional sensing dataset is transmitted to the edge computing module in real time. The edge computing module is used to execute the multi-sensor collaborative detection method of the wind turbine nacelle rail inspection robot according to any one of claims 1-6.

8. The multi-sensor collaborative detection system for a wind turbine nacelle rail-mounted inspection robot according to claim 7, characterized in that, The main controller uses a microcontroller or DSP chip, and it synchronizes the clock signal to the vibration sensor, infrared thermal imaging sensor and electromagnetic sensor via CAN bus, Ethernet PTP synchronization protocol or UART bus.

9. The multi-sensor collaborative detection system for a wind turbine nacelle rail-mounted inspection robot according to claim 7, characterized in that, The vibration sensor is a piezoelectric vibration sensor or a capacitive vibration sensor; the infrared thermal imaging sensor is a non-contact infrared thermal imaging sensor or a cooled infrared thermal imaging sensor; and the electromagnetic sensor is a Hall effect electromagnetic sensor or an inductive electromagnetic sensor.

10. A multi-sensor collaborative detection system for a wind turbine nacelle rail-mounted inspection robot, characterized in that, include: Memory, used to store instructions; The processor is configured to read instructions stored in the memory and execute the multi-sensor collaborative detection method for the wind turbine nacelle rail inspection robot according to any one of claims 1-6.

Images