Methods, systems, electronic devices and storage media for identifying faults in marine propulsion shafting

Abstract

This application provides a method, system, electronic device, and storage medium for identifying faults in a ship's propulsion shafting, belonging to the field of intelligent fault identification. The method acquires multimodal monitoring signals from the ship's propulsion shafting, including multi-source vibration signals and sound signals. The multi-source vibration signals include single vibration signals from multiple monitoring locations. Spatiotemporal correlation analysis and weighted fusion are performed on the multi-source vibration signals to obtain a fused vibration signal, thereby enhancing the vibration signal and improving its reliability. A ship propulsion shafting fault identification model is then used to process the processed multimodal monitoring signals to obtain fault identification results. The model uses a dynamic routing algorithm to dynamically weight and fuse multimodal features, fully considering the importance of different modal signals, enhancing the representational ability of features, and improving the accuracy of ship propulsion shafting fault identification.

Description

Technical Field

[0001] This application relates to the field of intelligent fault identification, and in particular to a method, system, electronic device and storage medium for identifying faults in a ship propulsion shafting system. Background Technology

[0002] As a core component of a ship's power system, the propulsion shafting system is responsible for transmitting the power generated by the main engine to the propeller. Due to its harsh working environment and heavy load, the performance of the shafting equipment gradually deteriorates over time. Once the performance deteriorates to a critical point, it may lead to equipment failure, which in turn may cause system performance degradation, equipment collapse, or even system paralysis, resulting in economic losses, casualties, and environmental pollution.

[0003] Currently, fault identification methods for ship propulsion shafting mainly rely on traditional monitoring techniques. These methods are often time-consuming and fail to fully utilize modern diagnostic technologies based on deep learning, hindering real-time monitoring and rapid diagnosis. Furthermore, many fault identification methods in related technologies use single-mode signals, such as vibration, acoustic, or current signals, as input features for analysis. However, single-mode signals contain limited information and are easily affected by external noise and equipment vibration in the complex operating environment of ships, resulting in low diagnostic accuracy and insufficient model robustness.

[0004] To address the aforementioned issues, although some studies have attempted to use multimodal signals for fault identification, they have failed to consider the sensitivity of different monitoring signals to different fault types, resulting in low accuracy in identifying faults in ship propulsion shafting systems. Summary of the Invention

[0005] The main objective of this application is to provide a method, system, electronic device, and storage medium for identifying faults in ship propulsion shafting, with the aim of improving the accuracy of fault identification in ship propulsion shafting.

[0006] To achieve the above objectives, one aspect of this application proposes a method for identifying faults in a ship's propulsion shafting system, comprising:

[0007] Acquire multimodal monitoring signals of a ship's propulsion shafting system, wherein the multimodal monitoring signals include multi-source vibration signals and sound signals, and the multi-source vibration signals include single vibration signals from multiple different monitoring locations;

[0008] Spatiotemporal correlation analysis and weighted fusion are performed on the multi-source vibration signals to obtain a fused vibration signal, and the multi-source vibration signals in the multi-modal monitoring signals are replaced with the fused vibration signal;

[0009] The multimodal monitoring signals are input into the ship propulsion shafting fault identification model to obtain the fault identification results of the ship propulsion shafting.

[0010] The ship propulsion shafting fault identification model specifically performs the following steps:

[0011] Feature extraction is performed on each modal signal of the multimodal monitoring signal to obtain the feature vector of each modality;

[0012] The fault identification result is obtained by dynamically weighting and fusing the feature vectors of each mode based on the dynamic routing algorithm.

[0013] In some embodiments, before the step of performing spatiotemporal correlation analysis and weighted fusion of the multi-source vibration signals to obtain a fused vibration signal, the ship propulsion shafting fault identification method further includes the following steps:

[0014] Each signal in the multi-source vibration signal is subjected to noise reduction processing to obtain the noise-reduced multi-mode monitoring signal;

[0015] The denoised multimodal monitoring signal is normalized to obtain the normalized multimodal monitoring signal.

[0016] In some embodiments, the step of performing spatiotemporal correlation analysis and weighted fusion on the multi-source vibration signals to obtain a fused vibration signal includes the following steps:

[0017] Spatiotemporal correlation analysis is performed on the multi-source vibration signals to determine the first weight of each individual vibration signal;

[0018] The multi-source vibration signals are weighted and fused according to the first weight of each individual vibration signal to obtain a fused vibration signal.

[0019] In some embodiments, determining the first weight of each individual vibration signal by performing spatiotemporal correlation analysis on the multi-source vibration signals includes the following steps:

[0020] The time deviation between each individual vibration signal is determined based on the signal source position of each individual vibration signal in the multi-source vibration signal.

[0021] Each single vibration signal is corrected according to the time deviation to obtain a time-synchronized single vibration signal;

[0022] Calculate the cross-correlation energy between each pair of individual vibration signals after time synchronization;

[0023] The total correlation energy of a single vibration signal is determined based on the multiple cross-correlation energies of each single vibration signal.

[0024] The first weight of each individual vibration signal is determined based on the total correlation energy of each individual vibration signal.

[0025] In some embodiments, determining the time deviation between individual vibration signals based on the signal source positions of each individual vibration signal in the multi-source vibration signal includes the following steps:

[0026] Construct a spatial coordinate matrix based on the signal source locations of multiple single vibration signals;

[0027] The target coordinate system is determined based on the signal source position of one of the single vibration signals, and the signal source positions of each of the single vibration signals are transformed to the target coordinate system based on the spatial coordinate matrix to obtain the position transformation matrix of each single vibration signal.

[0028] Based on the position transformation matrix of each individual vibration signal, the time deviation between the individual vibration signal and the individual vibration signal corresponding to the target coordinate system is determined.

[0029] In some embodiments, the multimodal monitoring signal includes a fused vibration signal and a sound signal. The step of extracting features from each modal signal of the multimodal monitoring signal to obtain the feature vector of each modality includes the following steps:

[0030] The fused vibration signal is input into a first-time convolutional neural network module for feature extraction to obtain a vibration feature vector.

[0031] The sound signal is input into a second temporal convolutional neural network module for feature extraction to obtain a sound feature vector.

[0032] Both the first temporal convolutional neural network module and the second temporal convolutional neural network module include multiple temporal convolutional blocks with residual connections.

[0033] In some embodiments, the step of dynamically weighting and fusing the feature vectors of each modality based on a dynamic routing algorithm to obtain the fault identification result includes the following steps:

[0034] The feature vectors of each mode are reconstructed separately to obtain the feature reconstruction vectors of each mode.

[0035] A dynamic routing algorithm is used to dynamically assign modal weights to the feature reconstruction vectors of each modality, thereby obtaining the second weights of each modality;

[0036] The multiple feature reconstruction vectors are weighted according to the second weight of each modality to obtain a prediction vector, wherein the prediction vector is used to determine the fault identification result.

[0037] To achieve the above objectives, another aspect of this application proposes a ship propulsion shafting fault identification system, comprising:

[0038] The first module is used to acquire multimodal monitoring signals of the ship's propulsion shafting system, wherein the multimodal monitoring signals include multi-source vibration signals and sound signals, and the multi-source vibration signals include single vibration signals from multiple different monitoring locations;

[0039] The second module is used to perform spatiotemporal correlation analysis and weighted fusion on the multi-source vibration signals to obtain a fused vibration signal, and to replace the multi-source vibration signals in the multi-modal monitoring signals with the fused vibration signal;

[0040] The third module is used to input the multimodal monitoring signals into the ship propulsion shafting fault identification model to obtain the fault identification results of the ship propulsion shafting.

[0041] Specifically, the ship propulsion shafting fault identification model is used for:

[0042] Feature extraction is performed on each modal signal of the multimodal monitoring signal to obtain the feature vector of each modality;

[0043] The fault identification result is obtained by dynamically weighting and fusing the feature vectors of each mode based on the dynamic routing algorithm.

[0044] To achieve the above objectives, another aspect of the present application provides an electronic device, which includes a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for enabling communication between the processor and the memory. When the program is executed by the processor, it implements the method described in the above embodiments.

[0045] To achieve the above objectives, another aspect of the embodiments of this application proposes a storage medium, which is a computer-readable storage medium for computer-readable storage. The storage medium stores one or more programs that can be executed by one or more processors to implement the methods described in the above embodiments.

[0046] The method, system, electronic device, and storage medium for identifying ship propulsion shafting faults proposed in this application acquire multimodal monitoring signals of the ship propulsion shafting. These signals include multi-source vibration signals and sound signals. The multi-source vibration signals include single vibration signals from multiple monitoring locations. Spatiotemporal correlation analysis and weighted fusion of the multi-source vibration signals are performed to obtain a fused vibration signal, thereby enhancing the vibration signal and improving its reliability. The processed multimodal monitoring signals are then processed using a ship propulsion shafting fault identification model to obtain fault identification results. The model uses a dynamic routing algorithm to dynamically weight and fuse multimodal features, fully considering the importance of different modal signals, enhancing the representation ability of features, and improving the accuracy of ship propulsion shafting fault identification. Attached Figure Description

[0047] Figure 1 This is a flowchart of the ship propulsion shafting fault identification method provided in the embodiments of this application;

[0048] Figure 2 This is a flowchart of the ship propulsion shafting fault identification model processing method provided in the embodiments of this application;

[0049] Figure 3 This is a schematic diagram of the ship propulsion shafting fault identification model provided in the embodiments of this application;

[0050] Figure 4 This is a schematic diagram of the TCN residual block structure provided in an embodiment of this application;

[0051] Figure 5 This is a schematic diagram of the network architecture of the multimodal feature weighted fusion module provided in the embodiments of this application;

[0052] Figure 6 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0053] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0054] It should be noted that although the system is divided into functional modules and the flowchart shows a logical order, in some cases, the steps shown or described may be executed in a different order than the module division in the system or the order in the flowchart. The terms "first," "second," etc., in the specification, claims, and the aforementioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0055] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0056] This application provides a method, system, electronic device, and storage medium for identifying faults in a ship's propulsion shafting system, aiming to improve the accuracy of fault identification in a ship's propulsion shafting system.

[0057] The ship propulsion shafting fault identification method, system, electronic device, and storage medium provided in this application are specifically described through the following embodiments. First, the ship propulsion shafting fault identification method in this application embodiment is described.

[0058] The ship propulsion shafting fault identification method provided in this application relates to the field of intelligent fault identification. This method can be applied to a terminal, a server, or software running on either a terminal or a server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, etc.; the server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the ship propulsion shafting fault identification method, but is not limited to the above forms.

[0059] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0060] Figure 1 This is an optional flowchart of the ship propulsion shafting fault identification method provided in the embodiments of this application. Figure 1 The method may include, but is not limited to, steps S101 to S103.

[0061] Step S101: Acquire multi-modal monitoring signals of the ship's propulsion shafting system. The multi-modal monitoring signals include multi-source vibration signals and sound signals. The multi-source vibration signals include single vibration signals from multiple different monitoring locations.

[0062] Step S102: Perform spatiotemporal correlation analysis and weighted fusion on the multi-source vibration signals to obtain the fused vibration signal, and replace the multi-source vibration signals in the multi-modal monitoring signals with the fused vibration signal;

[0063] Step S103: Input the multimodal monitoring signal into the ship propulsion shaft system fault identification model to obtain the fault identification result of the ship propulsion shaft system.

[0064] In step S101 of some embodiments, multiple single vibration signals of the ship's propulsion shaft system are collected by multiple vibration sensors arranged at different monitoring locations to form a multi-source vibration signal. Sound signals of the ship's propulsion shaft system are collected by acoustic sensors. The multi-modal monitoring signal of this embodiment includes multi-source vibration signals of vibration modes and sound signals of acoustic modes. It is understood that the multi-modal monitoring signal of this application embodiment can also include signals of more different types of modes based on actual monitoring needs.

[0065] In step S102 of some embodiments, spatiotemporal correlation analysis is a method for studying the relationship between data in time and space. In this embodiment, monitoring signals of the ship's propulsion shaft system are continuously collected. The monitoring signals are time-series signals with temporal attributes, while the multi-source vibration signals are composed of vibration signals from sensors at different locations and have spatial attributes. By fusing the multi-source vibration signals based on spatiotemporal correlation analysis, the vibration signals are enhanced, and the reliability of the vibration modes is improved.

[0066] In step S103 of some embodiments, this embodiment constructs a ship propulsion shafting fault identification model. This model is a classification model used to extract and classify features based on the input multimodal monitoring signals, and output the fault identification results of the ship propulsion shafting. The fault identification results are used to determine the state type label of the ship propulsion shafting, such as no fault and fault, and the fault type under fault condition. The ship propulsion shafting fault identification model includes two parts: feature extraction and feature classification. The feature extraction part can be implemented based on a temporal convolutional neural network (TCN). The temporal convolutional neural network can analyze the temporal features in the input data and extract features of various modes such as vibration and sound, capture the long-term dependence of modal temporal signals, and mine early fault features. Feature classification can be implemented using neural networks based on dynamic routing algorithms. Dynamic routing algorithms are mechanisms for allocating information and determining information transmission paths, and are particularly suitable for new neural network architectures such as capsule networks. In neural networks, dynamic routing algorithms are mainly used to dynamically determine how to allocate information to different network parts or paths based on the characteristics of the input data, thereby improving the adaptability to different modal features. For example, the feature classification part uses dynamic routing algorithms between capsules.

[0067] The ship propulsion shafting fault identification model in this embodiment is trained in the following way:

[0068] The dataset is divided into training and testing sets according to a preset ratio. The dataset contains a large number of samples. Each sample includes a multimodal monitoring signal after vibration signal fusion processing and a sample label. The sample label is the ship propulsion shafting status corresponding to the multimodal monitoring signal, such as no fault, faulty, and fault type.

[0069] After training the initialized ship propulsion shaft system fault identification model using the training set, the trained model is tested using the test set to obtain an applicable ship propulsion shaft system fault identification model.

[0070] The specific process of training the initialized ship propulsion shafting fault identification model with the training set is as follows:

[0071] The training set is input into the initialized ship propulsion shafting fault identification model to obtain the predicted label vector;

[0072] The accuracy of the ship propulsion shafting fault identification model is calculated based on the predicted label vector and the actual label vector.

[0073] If the recognition accuracy is less than the threshold accuracy, the model parameters are adjusted and the model is trained again until the number of training iterations exceeds the threshold number of iterations or the recognition accuracy after training is greater than or equal to the threshold accuracy. At this point, the trained ship propulsion shaft system fault recognition model is obtained.

[0074] In some embodiments, prior to step S102, the ship propulsion shafting fault identification method of this application may also include, but is not limited to, the following steps:

[0075] Step S301: Denoise each signal in the multi-source vibration signal to obtain the denoised multi-mode monitoring signal;

[0076] Step S302: Normalize the denoised multimodal monitoring signal to obtain the normalized multimodal monitoring signal.

[0077] In this embodiment, each signal from the multi-source vibration signal can be input into a suitable signal filter for noise reduction processing to remove noise and outliers from the data. Then, the multimodal monitoring signal is normalized to bring data with different characteristics to the same level, facilitating subsequent model calculations and improving model convergence speed. Specifically, the individual vibration signals of the multi-source vibration signal can be normalized first, and then the modal signals of the multimodal monitoring signal can be normalized.

[0078] Please refer to Figure 2 The specific steps for the ship propulsion shaft system fault identification model are as follows:

[0079] Step S201: Extract features from each modal signal of the multimodal monitoring signal to obtain the feature vector of each modality;

[0080] Step S202: Based on the dynamic routing algorithm, the feature vectors of each mode are dynamically weighted and fused to obtain the fault identification result.

[0081] In this embodiment, after preprocessing operations such as noise reduction and normalization are performed on the multimodal monitoring signals, and the multi-source vibration signals in the multimodal monitoring signals are processed into fused vibration signals, the multimodal monitoring signals are input into the ship propulsion shafting fault identification model to obtain the fault identification results of the ship propulsion shafting. The ship propulsion shafting fault identification model uses a dynamic routing algorithm to dynamically weight and fuse multimodal features, fully considering the importance of different modal signals, enhancing the representation ability of features, and improving the accuracy of fault identification.

[0082] In some embodiments, step S102, which involves performing spatiotemporal correlation analysis and weighted fusion on the multi-source vibration signals to obtain a fused vibration signal, may include, but is not limited to, the following steps:

[0083] Step S401: Perform spatiotemporal correlation analysis on the multi-source vibration signals to determine the first weight of each individual vibration signal;

[0084] Step S402: The multi-source vibration signals are weighted and fused according to the first weight of each individual vibration signal to obtain the fused vibration signal.

[0085] In step S401 of some embodiments, the step of performing spatiotemporal correlation analysis on the multi-source vibration signals to determine the first weight of each individual vibration signal may include, but is not limited to, the following steps:

[0086] Step S501: Determine the time deviation between each individual vibration signal based on the signal source position of each individual vibration signal in the multi-source vibration signal.

[0087] Step S502: Correct each single vibration signal according to the time deviation to obtain a time-synchronized single vibration signal;

[0088] Step S503: Calculate the cross-correlation energy between each pair of individual vibration signals after time synchronization;

[0089] Step S504: Determine the total correlation energy of a single vibration signal based on the multiple cross-correlation energies of each single vibration signal;

[0090] Step S505: Determine the first weight of each individual vibration signal based on the total correlation energy of each individual vibration signal.

[0091] In this embodiment, firstly, based on the signal source positions of each individual vibration signal in the multi-source vibration signal, the time deviation between each individual vibration signal is determined. For example, in this embodiment, three vibration sensors can be used to collect three individual vibration signals, namely vibration signal 1, vibration signal 2, and vibration signal 3. One of the individual vibration signals (vibration signal 1) can be used as a time reference to calculate the time deviation between the other two individual vibration signals and that individual vibration signal, thus obtaining the time deviation between each individual vibration signal. The time deviation of vibration signal 1 is 0, the time deviation between vibration signal 2 and vibration signal 1 is τ1, and the time deviation between vibration signal 3 and vibration signal 1 is τ2.

[0092] Secondly, each individual vibration signal is corrected based on the time deviation to obtain a time-synchronized individual vibration signal. For example, considering the time deviation, the corrected vibration signal 2 is expressed as y(t+τ1)=y(t)·δ(τ1), and the corrected vibration signal 3 is expressed as z(t+τ2)=z(t)·δ(τ2), where δ(t) is the Dirac function, used to represent the time shift of the signal. The time synchronization signals of the three vibration sensors are x(t), y(t+τ1), and z(t+τ2), respectively.

[0093] Then, the cross-correlation energy between each pair of individual vibration signals after time synchronization is calculated. For example, the correlation energy between discrete signals x(t) and y(t+τ1) is calculated using the following formula:

[0094]

[0095] Among them, R xy is the cross-correlation function of the vibration signal, where i represents the ordinal number of the cross-correlation function. The formula for calculating the cross-correlation function is as follows:

[0096]

[0097] Where N is the number of sampling points, k represents the maximum time delay value, and m represents the index variable of the time delay, which varies in the range of 1-k.

[0098] Assuming that cross-correlation is performed on each signal, the energy of the signal is E. ij The total correlation energy between a single vibration signal i and the single vibration signal i is expressed as follows:

[0099]

[0100] Where i represents the symbol of a single vibration signal, j represents a signal other than the single vibration signal i, and i ≠ j.

[0101] Suppose there are three single vibration signals, when the weight p of the single vibration signal i is... iWhen the energy is proportional to the correlation function, then:

[0102] p1:p2:p3=E1:E2:E3;

[0103] Normalization yields:

[0104] p1 + p2 + p3 = 1;

[0105] Based on the above operations, the first weight of the vibration sensor signal at each location can be obtained, as follows:

[0106]

[0107] In step S402 of some embodiments, a weighted calculation is performed based on the first weight of each individual vibration signal to obtain a fused vibration signal, which is represented as:

[0108] X = p1x + p2Y + p3z;

[0109] Where x, y, and z are the single vibration signals from the three vibration sensors, respectively.

[0110] At this point, the weighted fusion of multi-source vibration signals is completed, and the weighted fused signal is obtained.

[0111] In some embodiments, step S501, which determines the time deviation between individual vibration signals based on the signal source positions of each individual vibration signal in the multi-source vibration signal, may include, but is not limited to, the following steps:

[0112] Step S601: Construct a spatial coordinate matrix based on the signal source locations of multiple single vibration signals;

[0113] Step S602: Determine the target coordinate system based on the signal source position of one of the single vibration signals, and transform the signal source positions of each single vibration signal to the target coordinate system based on the spatial coordinate matrix to obtain the position transformation matrix of each single vibration signal.

[0114] Step S603: Determine the time deviation between the single vibration signal and the single vibration signal corresponding to the target coordinate system based on the position transformation matrix of each single vibration signal.

[0115] In this embodiment,

[0116] A spatial coordinate system is established with the signal source as the origin, yielding the spatial coordinates of multiple sensors. This embodiment, based on common mechanical system vibration sensor placement principles, proposes using three vibration sensors (horizontal, vertical, and at arbitrary positions) to monitor the vibration of the ship's propulsion shaft system. The spatial coordinate matrix of the three vibration sensors (i.e., the signal source) is then:

[0117]

[0118] Where X1, X2, and X3 represent the horizontal coordinates of the three vibration sensors; Y1, Y2, and Y3 represent the vertical coordinates of the three vibration sensors; Z1, Z2, and Z3 represent the vertical coordinates of the three vibration sensors; and b1, c1, a1, c2, a3, b3, and c3 represent the specific coordinate values.

[0119] To eliminate time discrepancies between the three vibration sensors, the spatial information from the three sensors is unified into the coordinate system of sensor 1. The formula is as follows:

[0120]

[0121] Spatial location correlation is used to estimate the time deviation between two sensors. For example, using sensor 1 as a reference, the time deviations of sensors 2 and 3 relative to sensor 1 are expressed as follows:

[0122]

[0123] Wherein, τ1 and τ2 represent the time deviations of sensor 2 and sensor 3 relative to sensor 1, respectively, that is, the time deviations of single vibration signal 2 and single vibration signal 3 relative to single vibration signal 1.

[0124] Substituting the coordinate values, we get:

[0125]

[0126] In some embodiments, the multimodal monitoring signal includes a fused vibration signal and a sound signal. Step S201, which involves extracting features from each modal signal of the multimodal monitoring signal to obtain the feature vector of each modality, may include, but is not limited to, the following steps:

[0127] Step S701: Input the fused vibration signal into the first time convolutional neural network module for feature extraction to obtain the vibration feature vector;

[0128] Step S702: Input the sound signal into the second temporal convolutional neural network module for feature extraction to obtain the sound feature vector;

[0129] Both the first-time convolutional neural network module and the second-time convolutional neural network module include multiple temporal convolutional blocks with residual connections.

[0130] Please refer to Figure 3 The fused vibration signal and the preprocessed sound signal are input separately into two parallel TCN modules (i.e., temporal convolutional neural network modules) for feature extraction and temporal feature analysis.

[0131] The TCN module consists of multiple TCN residual blocks (i.e., temporal convolutional blocks with residual connections). Please refer to [the relevant documentation / reference]. Figure 4 The TCN residual block consists of a main path and a residual connection. The main path includes an extended causal convolutional layer, a weight normalization layer, a ReLU activation function, a Dropout layer, etc., which are used to extract features and perform temporal feature analysis on the input time-series signal. The path of the residual connection contains a 1*1 convolutional layer to make up for the difference between the input and output dimensions, and at the same time capture the long-term dependencies of the time-series signal.

[0132] After the signal passes through multiple TCN residual blocks, the outputs of the main path and residual connections in the TCN residual blocks are added together to obtain vector outputs of various modal characteristics such as vibration and acoustics.

[0133] In some embodiments, step S202, which involves dynamically weighting and fusing the feature vectors of each modality based on a dynamic routing algorithm to obtain the fault identification result, may include, but is not limited to, the following steps:

[0134] Step S801: Perform feature reconstruction on the feature vectors of each mode to obtain the feature reconstruction vectors of each mode;

[0135] Step S802: The dynamic routing algorithm is used to perform modal dynamic weight allocation on the feature reconstruction vectors of each modality to obtain the second weight of each modality;

[0136] Step S803: The multiple feature reconstruction vectors are weighted according to the second weight of each modality to obtain the prediction vector, which is used to determine the fault identification result.

[0137] In this embodiment, please continue to refer to Figure 3This embodiment constructs a multimodal feature weighted fusion module based on a capsule network structure to fuse and analyze input features to obtain a prediction vector. The multimodal feature weighted fusion module includes a main capsule layer, a dynamic routing algorithm module, and a conditional capsule layer. The main capsule layer reconstructs the feature vectors of each modality, obtaining feature reconstruction vectors for each modality, such as sound and vibration. Through feature reconstruction, features with stronger correlation to the target variable can be generated, thereby improving the model's predictive performance and generalization ability. The reconstructed features can better capture the patterns in the data, making the model more accurate and stable. The dynamic routing algorithm module dynamically assigns modal weights to the feature reconstruction vectors of each modality output from the main capsule layer, obtaining the second weights for each modality, and outputs them to the conditional capsule layer. The conditional capsule layer performs a weighted summation of multiple feature reconstruction vectors based on the second weights of each modality, outputting a prediction vector. The output prediction vector has magnitude and direction; the direction represents different fault labels, and its magnitude represents the probability of different fault labels. The fault type represented by the vector with the largest magnitude is selected as the fault identification result for the ship's propulsion shaft system.

[0138] Specifically, the network architecture of the multimodal feature weighted fusion module is as follows: Figure 5 As shown, its implementation process is as follows:

[0139] The modal vectors output by the temporal convolutional neural network module are input into the main capsule layer for reconstruction, resulting in the reconstructed vector P. i ;

[0140] Vector P i Squash compression yields S i The compression operation is represented as follows:

[0141]

[0142] Construct a transformation matrix for the input vector S i The transformation is performed as follows: S′ i =W i ·S i S i W is the transformed vector. i Let i be the transformation matrix, and let i be the current capsule number in the main capsule layer.

[0143] Initialize the temporary variable of coupling coefficient Where j is the current capsule number in the conditional capsule layer;

[0144] The Softmax function is used to calculate the weights between the master capsule and the conditional capsules. The specific calculation method for the weights is as follows:

[0145]

[0146] Where k is the total number of condition capsules;

[0147] Calculate the prediction vector D for all conditional capsules. j Prediction vector D j The specific calculations are as follows:

[0148]

[0149] Update the temporary variable of coupling coefficient The update process is as follows:

[0150] The weights are updated iteratively until a predefined number of routes is reached, resulting in the dynamically weighted and fused output vector D. j .

[0151] This application also proposes a ship propulsion shafting fault identification system, including:

[0152] The first module is used to acquire multimodal monitoring signals of the ship's propulsion shafting system. The multimodal monitoring signals include multi-source vibration signals and sound signals. The multi-source vibration signals include single vibration signals from multiple different monitoring locations.

[0153] The second module is used to perform spatiotemporal correlation analysis and weighted fusion of multi-source vibration signals to obtain a fused vibration signal, and to replace the multi-source vibration signals in the multi-modal monitoring signals with the fused vibration signal.

[0154] The third module is used to input multimodal monitoring signals into the ship propulsion shafting fault identification model to obtain the fault identification results of the ship propulsion shafting.

[0155] Specifically, the ship propulsion shafting fault identification model is used for:

[0156] Feature extraction is performed on each modal signal of the multimodal monitoring signal to obtain the feature vector of each mode;

[0157] The fault identification result is obtained by dynamically weighting and fusing the feature vectors of each mode based on the dynamic routing algorithm.

[0158] It is understood that the content of the above embodiments of the ship propulsion shafting fault identification method is applicable to this system embodiment. The specific functions implemented in this system embodiment are the same as those in the above embodiments of the ship propulsion shafting fault identification method, and the beneficial effects achieved are also the same as those achieved in the above embodiments of the ship propulsion shafting fault identification method.

[0159] This application also provides an electronic device, which includes: a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for communication between the processor and the memory. When the program is executed by the processor, it implements the aforementioned method for identifying faults in a ship propulsion shaft system. This electronic device can be any smart terminal, including tablet computers, in-vehicle computers, etc.

[0160] Please see Figure 6 , Figure 6 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes:

[0161] The processor 601 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application.

[0162] The memory 602 can be implemented as a read-only memory (ROM), static storage device, dynamic storage device, or random access memory (RAM). The memory 602 can store the operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 602 and is called and executed by the processor 601 to execute the ship propulsion shaft fault identification method of the embodiments of this application.

[0163] The input / output interface 603 is used to implement information input and output;

[0164] The communication interface 604 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0165] Bus 605 transmits information between various components of the device (e.g., processor 601, memory 602, input / output interface 603, and communication interface 604);

[0166] The processor 601, memory 602, input / output interface 603, and communication interface 604 are connected to each other within the device via bus 605.

[0167] This application also provides a storage medium, which is a computer-readable storage medium for computer-readable storage. The storage medium stores one or more programs, which can be executed by one or more processors to implement the above-described ship propulsion shaft fault identification method.

[0168] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0169] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0170] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0171] The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0172] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0173] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0174] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0175] In the embodiments provided in this application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between systems or units may be electrical, mechanical, or other forms.

[0176] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0177] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0178] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0179] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A method for identifying faults in a ship's propulsion shafting system, characterized in that, Includes the following steps: Acquire multimodal monitoring signals of a ship's propulsion shafting system, wherein the multimodal monitoring signals include multi-source vibration signals and sound signals, and the multi-source vibration signals include single vibration signals from multiple different monitoring locations; Spatiotemporal correlation analysis and weighted fusion are performed on the multi-source vibration signals to obtain a fused vibration signal, and the multi-source vibration signals in the multi-modal monitoring signals are replaced with the fused vibration signal; The multimodal monitoring signals are input into the ship propulsion shafting fault identification model to obtain the fault identification results of the ship propulsion shafting. The ship propulsion shafting fault identification model specifically performs the following steps: Feature extraction is performed on each modal signal of the multimodal monitoring signal to obtain the feature vector of each modality; The fault identification result is obtained by dynamically weighting and fusing the feature vectors of each mode based on the dynamic routing algorithm. The multimodal monitoring signal includes a fused vibration signal and a sound signal. The step of extracting features from each modal signal of the multimodal monitoring signal to obtain the feature vector of each modality includes the following steps: The fused vibration signal is input into a first-time convolutional neural network module for feature extraction to obtain a vibration feature vector. The sound signal is input into a second temporal convolutional neural network module for feature extraction to obtain a sound feature vector. Both the first temporal convolutional neural network module and the second temporal convolutional neural network module include multiple temporal convolutional blocks with residual connections.

2. The method for identifying faults in a ship propulsion shafting system according to claim 1, characterized in that, Before the step of performing spatiotemporal correlation analysis and weighted fusion on the multi-source vibration signals to obtain the fused vibration signal, the ship propulsion shafting fault identification method further includes the following steps: Each signal in the multi-source vibration signal is subjected to noise reduction processing to obtain the noise-reduced multi-mode monitoring signal; The denoised multimodal monitoring signal is normalized to obtain the normalized multimodal monitoring signal.

3. The method for identifying faults in a ship propulsion shafting system according to claim 1, characterized in that, The process of performing spatiotemporal correlation analysis and weighted fusion on the multi-source vibration signals to obtain the fused vibration signal includes the following steps: Spatiotemporal correlation analysis is performed on the multi-source vibration signals to determine the first weight of each individual vibration signal; The multi-source vibration signals are weighted and fused according to the first weight of each individual vibration signal to obtain a fused vibration signal.

4. The method for identifying faults in a ship propulsion shafting system according to claim 3, characterized in that, The step of performing spatiotemporal correlation analysis on the multi-source vibration signals to determine the first weight of each individual vibration signal includes the following steps: The time deviation between each individual vibration signal is determined based on the signal source position of each individual vibration signal in the multi-source vibration signal. Each single vibration signal is corrected according to the time deviation to obtain a time-synchronized single vibration signal; Calculate the cross-correlation energy between each pair of individual vibration signals after time synchronization; The total correlation energy of a single vibration signal is determined based on the multiple cross-correlation energies of each single vibration signal. The first weight of each individual vibration signal is determined based on the total correlation energy of each individual vibration signal.

5. The method for identifying faults in a ship propulsion shafting system according to claim 4, characterized in that, Determining the time deviation between individual vibration signals based on the signal source positions of each individual vibration signal in the multi-source vibration signal includes the following steps: Construct a spatial coordinate matrix based on the signal source locations of multiple single vibration signals; The target coordinate system is determined based on the signal source position of one of the single vibration signals, and the signal source positions of each of the single vibration signals are transformed to the target coordinate system based on the spatial coordinate matrix to obtain the position transformation matrix of each single vibration signal. Based on the position transformation matrix of each individual vibration signal, the time deviation between the individual vibration signal and the individual vibration signal corresponding to the target coordinate system is determined.

6. The method for identifying faults in a ship propulsion shafting system according to claim 1, characterized in that, The method of dynamically weighting and fusing the feature vectors of each modality based on the dynamic routing algorithm to obtain the fault identification result includes the following steps: The feature vectors of each mode are reconstructed separately to obtain the feature reconstruction vectors of each mode. A dynamic routing algorithm is used to dynamically assign modal weights to the feature reconstruction vectors of each modality, thereby obtaining the second weights of each modality; The multiple feature reconstruction vectors are weighted according to the second weight of each modality to obtain a prediction vector, wherein the prediction vector is used to determine the fault identification result.

7. A fault identification system for ship propulsion shafting, characterized in that, include: The first module is used to acquire multimodal monitoring signals of the ship's propulsion shafting system, wherein the multimodal monitoring signals include multi-source vibration signals and sound signals, and the multi-source vibration signals include single vibration signals from multiple different monitoring locations; The second module is used to perform spatiotemporal correlation analysis and weighted fusion on the multi-source vibration signals to obtain a fused vibration signal, and to replace the multi-source vibration signals in the multi-modal monitoring signals with the fused vibration signal; The third module is used to input the multimodal monitoring signals into the ship propulsion shafting fault identification model to obtain the fault identification results of the ship propulsion shafting. Specifically, the ship propulsion shafting fault identification model is used for: Feature extraction is performed on each modal signal of the multimodal monitoring signal to obtain the feature vector of each modality; The fault identification result is obtained by dynamically weighting and fusing the feature vectors of each mode based on the dynamic routing algorithm. The multimodal monitoring signal includes a fused vibration signal and a sound signal. The step of extracting features from each modal signal of the multimodal monitoring signal to obtain the feature vector of each modality includes the following steps: The fused vibration signal is input into a first-time convolutional neural network module for feature extraction to obtain a vibration feature vector. The sound signal is input into a second temporal convolutional neural network module for feature extraction to obtain a sound feature vector. Both the first temporal convolutional neural network module and the second temporal convolutional neural network module include multiple temporal convolutional blocks with residual connections.

8. An electronic device, characterized in that, The electronic device includes a memory, a processor, a program stored in the memory and running on the processor, and a data bus for enabling communication between the processor and the memory, wherein the program, when executed by the processor, implements the steps of the method as described in any one of claims 1 to 6.

9. A storage medium, said storage medium being a computer-readable storage medium for computer-readable storage, characterized in that, The storage medium stores one or more programs, which are executed by one or more processors to implement the steps of the method according to any one of claims 1 to 6.

Images