Ship power equipment state monitoring and fault diagnosis method and system based on multi-source data visualization

Abstract

The invention provides a ship power system intelligent monitoring and early fault diagnosis method and system based on multi-modal image recognition. The method comprises the following steps: S1, constructing a three-dimensional digital twin base model of monitored power equipment; s2, multiple types of operation time sequence signals of the monitored power equipment are collected; s3, converting the acquired at least one operation time sequence signal into a two-dimensional feature image; s4, mapping and fusing the two-dimensional feature image and a parameterized data stream generated based on the operation time sequence signal to a corresponding spatial position of the three-dimensional digital twin base model, and generating a fused visual training picture used by a machine learning model; and S5, training a visual AI model by using the generated fusion visual training picture, and performing state recognition and fault diagnosis on the fusion visual training picture generated in real time by using the trained model. According to the invention, a complex equipment state monitoring problem is converted into a visual mode identification problem, and ship power equipment monitoring and early, accurate and automatic fault diagnosis are realized.

Description

Technical Field

[0001] This invention belongs to the field of ship operation and maintenance and fault diagnosis technology, specifically relating to a method and system for monitoring the status and diagnosing faults of ship power equipment based on multi-source data visualization. Background Technology

[0002] Marine power monitoring technology falls under the category of process control. Traditional power monitoring focuses on the power unit itself, such as steam turbines, gas turbines, and diesel engines, as well as related peripheral systems. Its main function is to perform regulation, control, and protection of the relevant units under specific operating conditions. However, with the increasingly prominent role and development needs of intelligent control and operation and maintenance technologies in marine power monitoring, the scope of traditional process control is no longer sufficient for new types of marine power monitoring.

[0003] Marine propulsion systems (such as main engines, gearboxes, bearings, and propellers) are the "heart" of a ship, and their operational reliability is crucial. Traditional propulsion monitoring and fault diagnosis methods mainly rely on the analysis of one-dimensional time-series signals such as vibration, pressure, temperature, and rotational speed. These methods depend on expert experience for feature extraction and threshold setting, and have the following drawbacks: Using data directly presents the problem of information abstraction, as raw one-dimensional time series or spectral data are highly abstract. Models need to learn complex, hidden spatiotemporal dependencies from a string of numbers, which is usually more difficult and requires more sophisticated model structures.

[0004] In addition, it is highly dependent on feature engineering. Although deep learning can automatically extract features, it often still requires a lot of signal processing knowledge and feature engineering (such as extracting kurtosis, entropy, etc.) to assist the model in terms of data dimensions. Otherwise, the model may have difficulty converging.

[0005] While some technologies have attempted to convert signals into images for processing, in the specific field of marine propulsion, how to systematically integrate multi-source data (physical images and signal images) and utilize advanced AI vision technologies (such as object detection and segmentation) to achieve precise positioning and quantitative analysis remains an unsolved technical challenge. Summary of the Invention

[0006] The purpose of this invention is to provide a method and system for monitoring the status and diagnosing faults of ship power equipment based on multi-source data visualization.

[0007] A method for monitoring the condition and diagnosing faults of ship propulsion equipment based on multi-source data visualization includes the following steps: S1, Construct a three-dimensional digital twin base model of the monitored power equipment; S2, Collect various types of runtime sequence signals from the monitored power equipment; S3, convert at least one acquired runtime sequence signal into a two-dimensional feature image; S4: Map and fuse the two-dimensional feature image and the parameterized data stream generated based on runtime sequence signals to the corresponding spatial position of the three-dimensional digital twin base model to generate a fused visual training image for use by the machine learning model; S5: Use the generated fused visual training image to train the visual AI model, and use the trained model to perform state recognition and fault diagnosis on the real-time generated fused visual training image.

[0008] Further, S1 specifically involves: scanning the physical device using an image acquisition device, and generating a three-dimensional mesh model and texture map of the monitored power device based on structure of motion recovery (SFM) or neural radiation field (NeRF) technology; and digitally identifying and spatially locating key monitoring components in the three-dimensional digital twin base model.

[0009] Furthermore, in step S3, the method for converting the runtime sequence signal into a two-dimensional feature image includes at least one of the following: Time-frequency analysis techniques are used to convert time-series signals into time-frequency diagrams. The time-series signal is converted into a recursive graph using the Recursive Graph (RP) technique; the time-series signal is converted into a two-dimensional image using Gram angle field (GAF) or Markov transfer field (MTF) techniques; and the time-series signal is subjected to spectral analysis to generate a spectrogram.

[0010] Further, S4 specifically includes: attaching the two-dimensional feature image as a dynamic texture to the surface of the corresponding component in the three-dimensional digital twin base model; At least one of the parameters of pressure, temperature, and flow rate is visualized on the corresponding pipeline or line of the three-dimensional digital twin base model through virtual streamlines of different colors, widths, or animation effects.

[0011] Furthermore, in S4, the fused visual training image is a two-dimensional image rendered from a preset perspective by the fused three-dimensional scene; the image is highly information-saturated and designed specifically for machine learning models to read, without considering direct interaction and readability by human operators.

[0012] Further, S5 specifically includes: S5.1, collecting historical data and generating a large number of fused visual training images with health status labels to form a training dataset; S5.2, using the training dataset to perform supervised training on a convolutional neural network (CNN) or a Vision Transformer model, enabling the model to learn visual patterns corresponding to different health states; S5.3, in real-time monitoring, inputting the real-time generated fused visual training images into the trained visual AI model, outputting fault classification results and confidence levels, and triggering an alert when the confidence level exceeds a threshold.

[0013] A system for monitoring and diagnosing the condition of ship power equipment based on multi-source data visualization includes: a 3D modeling module for constructing and processing the 3D digital twin base model; a data acquisition and preprocessing module for acquiring and multiplexing the runtime timing signals and performing preprocessing; a signal-to-image module for converting the preprocessed timing signals into the 2D feature images; an image fusion generation module for mapping the 2D feature images and parameterized data streams onto the 3D digital twin base model and rendering the fused visual training images; and an intelligent diagnosis module for storing and training the visual AI model and using the model to perform diagnostic analysis on the real-time images.

[0014] Furthermore, the fused visual training image generated by the image fusion generation module includes both the spatial structure information of the monitored power equipment and multi-source dynamic operation data information expressed through image textures and dynamic streamlines.

[0015] The beneficial effects of this invention are as follows: This invention combines the power monitoring system with AI vision technology, and connects the field of signal processing with the rapidly developing field of computer vision through the bridge of "visualization". This enables the use of the latter's most advanced and powerful models and tools to ultimately achieve more accurate, reliable and automated intelligent fault diagnosis. Attached Figure Description

[0016] Figure 1 This is a flowchart of the present invention. Detailed Implementation

[0017] The present invention will now be further described with reference to the accompanying drawings.

[0018] like Figure 1 As shown, a method for monitoring the condition and diagnosing faults of ship propulsion equipment based on multi-source data visualization includes: S1, Construct a three-dimensional digital twin base model of the monitored equipment. The purpose of this step is to provide a precise, mappable physical space carrier for dynamically generated data.

[0019] 1. Data Acquisition: Using high-resolution digital cameras, laser scanners, or video equipment, multi-angle and all-round image or video data are acquired of the target ship's power equipment (such as main engine, gearbox, bearing housing, etc.) and its peripheral pipelines and lines.

[0020] 2. 3D Model Reconstruction: The acquired image / video data is input into 3D reconstruction software or algorithms to automatically generate a high-precision 3D mesh model and texture map of the device. This model forms the "static base" for subsequent data visualization.

[0021] 3. Key Component Identification and Spatial Positioning: In the generated 3D model, key monitoring points (such as bearing housings, cylinder heads, inlet and outlet flanges, etc.) are digitally identified and spatially positioned. These positioning points will serve as "anchor points" for the dynamic data stream.

[0022] S2, Acquisition and Preprocessing of Multi-Source Operating Signals This step runs in parallel with S1 and is responsible for collecting real-time operating status data of the equipment.

[0023] 1. Signal Acquisition: Multiple types of time-series signal data are simultaneously acquired using sensors installed at the key monitoring points identified in Step 1, including but not limited to: Vibration signals: used to analyze faults such as mechanical shock, imbalance, and misalignment.

[0024] Temperature signal: Used to monitor problems such as overheating and poor lubrication.

[0025] Pressure signals are used to assess the operating status of fluid systems (such as fuel, lubricating oil, and cooling water).

[0026] Speed, current, voltage, etc.: used to characterize the overall operating condition of the equipment.

[0027] 2. Signal preprocessing: Standardize the acquired raw signals, including denoising, filtering, and normalization, to eliminate the influence of dimensions and improve data quality.

[0028] S3 converts the time-series signal into a two-dimensional feature image. This step is crucial for transforming abstract "data" into intuitive "images".

[0029] 1. Time-Frequency Analysis: For non-stationary signals (such as vibration signals), time-frequency analysis techniques such as Short-Time Fourier Transform (STFT) or Wavelet Transform (WT) are used to convert one-dimensional time-series signals into two-dimensional time-frequency spectrograms. In this spectrogram, time, frequency, and signal energy correspond to the horizontal axis, vertical axis, and pixel color (or brightness), respectively. For example, the characteristic frequency of bearing failure will appear as a continuous "bright line" in the time-frequency spectrogram.

[0030] 2. Other visualization methods: Recurrence plots (RP) transform the periodic and nonlinear dynamic characteristics of time-series signals into textured two-dimensional images.

[0031] Image standardization: All generated feature images are scaled to the same size (e.g., 224x224 pixels) and can be converted to grayscale or pseudocolor images as needed to form standardized model inputs.

[0032] S4 generates fused visual training images for machine learning. This step is the core innovation of the invention, which integrates the static 3D model with the dynamic feature image into a unified, machine-readable "monitoring screen".

[0033] 1. Dynamic data stream mapping: The two-dimensional feature image generated in step three is used as a dynamic texture and, according to the spatial positioning defined in step one, is attached to the corresponding component of the three-dimensional base model. For example, the vibration time-frequency diagram of the bearing housing is displayed on the outer surface of the bearing housing in the three-dimensional model.

[0034] The pressure, temperature, and other parameters in the pipeline are visualized in a 3D model using virtual streamlines of different colors, widths, and flow animations. For example, high temperature is represented by a red streamline, and normal temperature by a green streamline; high pressure is represented by a thick streamline, and low pressure by a thin streamline.

[0035] 2. Generate training footage: The system renders the aforementioned 3D scene, which integrates static models and dynamic data, into a 2D image based on preset viewpoints (which may include multiple fixed viewpoints, such as front view, side view, and sectional view).

[0036] Key point: This rendered screen is not designed for human operators, therefore the aesthetics and simplicity of traditional human-machine interfaces are irrelevant. Its sole purpose is to serve as input for machine learning models. Therefore, the screen can densely contain feature images and data streams from all monitoring points, forming a highly information-saturated "feature fusion map."

[0037] Repeat this process for data from different time points or different health states to generate massive image datasets with labels (such as "normal", "bearing failure", "imbalance"), which serve as the training and testing sets for the model.

[0038] S5, Vision-based Training and Fault Diagnosis This step utilizes mature computer vision models to learn from and diagnose the generated "visual training images".

[0039] The large number of "fused visual training images" and their corresponding labels generated in step four are input into the CNN model for supervised training. The model's task is to learn the visual features (such as specific texture patterns, color distribution, and spatial relationships) corresponding to different fault modes from these images.

[0040] In real-time monitoring, the system continuously collects sensor data at the current moment and executes S2 to S4 in real time to generate the current "real-time fused image", which is then input into the trained visual diagnostic model.

[0041] The model outputs a classification result (such as "normal" or "fault type") and a confidence score. When a fault mode is identified and the confidence score exceeds a set threshold, the system automatically triggers an alert and can highlight the location of the faulty component on a traditional human-machine interface.

[0042] A system for monitoring and diagnosing the condition of ship power equipment based on multi-source data visualization includes: a 3D modeling module for constructing and processing the 3D digital twin base model; a data acquisition and preprocessing module for acquiring and multiplexing the runtime timing signals and performing preprocessing; a signal-to-image module for converting the preprocessed timing signals into the 2D feature images; and an image fusion generation module for mapping the 2D feature images and parameterized data streams onto the 3D digital twin base model and rendering the fused visual training images. The image fusion generation module generates a fused visual training image, which includes the spatial structure information of the monitored power equipment and multi-source dynamic operation data information expressed through image textures and dynamic streamlines; the intelligent diagnosis module is used to store and train the visual AI model, and use the model to perform diagnostic analysis on the real-time image.

[0043] Example: Take the bearing fault diagnosis of a certain type of ship propulsion as an example.

[0044] First, a 3D scan of the unit body is performed to create a digital model that includes components such as the crankshaft, main bearings, and connecting rods.

[0045] Install vibration sensors on the main bearing housing to continuously collect vibration signals (X-axis, Y-axis).

[0046] Wavelet transform is performed on the vibration signal to generate a time-frequency graph. Under normal conditions, the time-frequency graph has a uniform texture, while when a peeling fault occurs, periodic visual features will appear at specific frequencies.

[0047] In the merged visuals, this time-frequency texture is applied to the surface of the main bearing housing of the 3D model. Simultaneously, lubricating oil pressure and temperature are displayed as colored streamlines within the lubricating oil lines.

[0048] Train a CNN model using thousands of such fused images labeled “normal” and “bearing peeling”.

[0049] After deployment, the system generates a fused image of the current state in real time. The model can quickly identify the "visual feature" patterns that appear on the bearing housing texture in the image, thereby diagnosing early bearing failures and issuing warnings.

[0050] As can be seen from the above specific implementation methods, the present invention transforms the complex problem of ship power system fault diagnosis into an image pattern recognition problem that computer vision excels at by means of "data visualization" and "multi-source information fusion", which significantly improves the automation, accuracy and interpretability of fault diagnosis.

Claims

1. A method for monitoring the condition and diagnosing faults of ship propulsion equipment based on multi-source data visualization, characterized in that, Includes the following steps: S1, Construct a three-dimensional digital twin base model of the monitored power equipment; S2, Collect various types of runtime sequence signals from the monitored power equipment; S3, convert at least one acquired runtime sequence signal into a two-dimensional feature image; S4 maps and fuses the two-dimensional feature image and the parameterized data stream generated based on runtime sequence signals to the corresponding spatial position of the three-dimensional digital twin base model, generating a fused visual training screen for machine learning models. S5: Train a visual AI model using the generated fused visual training images, and use the trained model to perform state recognition and fault diagnosis on the real-time generated fused visual training images.

2. The method for monitoring and diagnosing the condition of ship propulsion equipment based on multi-source data visualization according to claim 1, characterized in that, S1 specifically involves: scanning the physical device using an image acquisition device, and generating a three-dimensional mesh model and texture map of the monitored power device based on structure of motion recovery (SFM) or neural radiation field (NeRF) technology; and digitally identifying and spatially locating key monitoring components in the three-dimensional digital twin base model.

3. The method for monitoring and diagnosing the condition of ship propulsion equipment based on multi-source data visualization according to claim 1, characterized in that, In step S3, the method for converting the runtime sequence signal into a two-dimensional feature image includes at least one of the following: Time-frequency analysis techniques are used to convert time-series signals into time-frequency diagrams. The time-series signal is converted into a recursive graph using the Recursive Graph (RP) technique; the time-series signal is converted into a two-dimensional image using Gram angle field (GAF) or Markov transfer field (MTF) techniques; and the time-series signal is subjected to spectral analysis to generate a spectrogram.

4. The method for monitoring and diagnosing the condition of ship propulsion equipment based on multi-source data visualization according to claim 1, characterized in that, S4 specifically includes: attaching a two-dimensional feature image as a dynamic texture to the surface of the corresponding component in the three-dimensional digital twin base model; At least one of the parameters of pressure, temperature, and flow rate is visualized on the corresponding pipeline or line of the three-dimensional digital twin base model through virtual streamlines of different colors, widths, or animation effects.

5. The method for monitoring and diagnosing the condition of ship propulsion equipment based on multi-source data visualization according to claim 1, characterized in that, In S4, the fused visual training image is a two-dimensional image rendered from a preset perspective by fusing the three-dimensional scene; the image is highly information-saturated and designed specifically for machine learning models to read, without considering direct interaction and readability with human operators.

6. The method for monitoring and diagnosing the condition of ship propulsion equipment based on multi-source data visualization according to claim 1, characterized in that, S5 specifically includes: S5.1, collecting historical data and generating a large number of fused visual training images with health status labels to form a training dataset; S5.2, using the training dataset to perform supervised training on a convolutional neural network (CNN) or a Vision Transformer model, enabling the model to learn the visual patterns corresponding to different health states; S5.3, in real-time monitoring, inputting the real-time generated fused visual training images into the trained visual AI model, outputting fault classification results and confidence levels, and triggering an early warning when the confidence level exceeds a threshold.

7. A system for monitoring and diagnosing the condition of marine power equipment based on multi-source data visualization, based on the method for monitoring and diagnosing the condition of marine power equipment based on multi-source data visualization as described in any one of claims 1-6, characterized in that, include: A 3D modeling module is used to construct and process the 3D digital twin base model; The data acquisition and preprocessing module is used to acquire and multiplex the runtime sequence signals and perform preprocessing. The signal-to-image module is used to convert the preprocessed time-series signal into the two-dimensional feature image. The image fusion generation module is used to map the two-dimensional feature image and parameterized data stream onto the three-dimensional digital twin base model and render the fused visual training image. The intelligent diagnostic module is used to store and train the visual AI model, and to use the model to perform diagnostic analysis on real-time images.

8. A ship propulsion equipment condition monitoring and fault diagnosis system based on multi-source data visualization according to claim 7, characterized in that, The fused visual training image generated by the image fusion generation module includes both the spatial structure information of the monitored power equipment and multi-source dynamic operation data information expressed through image textures and dynamic streamlines.

Images