Port equipment operation state identification method and system, terminal and medium

Abstract

The present application relates to the technical field of multi-source monitoring, and specifically provides a port equipment operation state identification method, system, terminal and medium, comprising: acquiring a multi-source detection signal; using an improved adaptive Kalman filter to denoise the multi-source detection signal to obtain a sample signal; decomposing the sample signal into multi-scale frequency domain features through wavelet transform and Fourier transform, and performing convolution processing on the multi-scale frequency domain features to obtain multi-scale time-frequency features; using an LSTM and CNN based multi-task learning model to identify the residual life, fault type and health state corresponding to the multi-scale time-frequency features. The present application can effectively remove the noise of the multi-source detection signal, effectively utilize the detection signal, and improve the monitoring efficiency and real-time monitoring.

Description

Technical Field

[0001] This invention relates to the field of multi-source monitoring technology, specifically to a method, system, terminal, and medium for identifying the operational status of port equipment. Background Technology

[0002] The core of port equipment condition monitoring lies in the effective processing of signals from multiple sensors. However, in practice, the signal characteristics generated by different sensors vary greatly, making it difficult to adopt a unified processing method. Moreover, noise interference in the signals further increases the processing difficulty.

[0003] In traditional Kalman filtering applications, it is typically assumed that noise and system dynamics are constant. However, in complex port equipment, due to environmental factors, changes in equipment status, and other reasons, the dynamics and noise characteristics of the system may be dynamically changing. Therefore, traditional Kalman filtering cannot effectively denoise dynamic multi-source sensor signals.

[0004] Currently used threshold comparison monitoring methods have many limitations. On the one hand, their monitoring accuracy is limited, failing to fully extract useful information from sensor signals, leading to inaccurate judgments of equipment status. On the other hand, obtaining multiple monitoring results requires setting up multiple sets of judgment logic, which not only causes redundant calculations and wastes valuable computing resources, but also reduces data processing efficiency, making it difficult to meet the needs of real-time monitoring and hindering the timely detection of potential equipment failures. Summary of the Invention

[0005] To address the aforementioned shortcomings of existing technologies, this invention provides a method, system, terminal, and medium for identifying the operational status of port equipment, thereby resolving the aforementioned technical problems.

[0006] In a first aspect, the present invention provides a method for identifying the operational status of port equipment, comprising:

[0007] Acquire multi-source detection signals;

[0008] The multi-source detection signal is denoised using an improved adaptive Kalman filter to obtain a sample signal;

[0009] The sample signal is decomposed into multi-scale frequency domain features by wavelet transform and Fourier transform, and the multi-scale frequency domain features are convolved to obtain multi-scale time-frequency features.

[0010] The remaining lifetime, fault type, and health status corresponding to the multi-scale time-frequency features are identified using a multi-task learning model based on LSTM and CNN.

[0011] In one optional implementation, acquiring multi-source detection signals includes:

[0012] The detection signals of a sensor combination pre-deployed at multiple detection points in port equipment are acquired. The sensor combination includes a temperature sensor, a vibration sensor, a current sensor, a voltage sensor, a pressure sensor, and a displacement sensor.

[0013] In an optional implementation, the adaptive Kalman filter includes:

[0014] Adaptive process noise covariance :

[0015]

[0016] in, It is a smoothing factor. It is the state matrix of the sensor at time k. It is the variance of the current sensor state matrix;

[0017] Adaptive Measurement Noise Covariance :

[0018]

[0019] in, It is a smoothing factor. It is the detection signal matrix generated at time k. It is the variance of the current detection signal matrix.

[0020] In an optional implementation, the multi-source detection signal is denoised using an improved adaptive Kalman filter to obtain a sample signal, including:

[0021] Pre-set smoothing factor values ​​for various operating conditions, as well as the range of detection signal values;

[0022] The multi-source detection signal is matched with the detection signal value range corresponding to each working condition to determine the current working condition;

[0023] The smoothing factor value corresponding to the current operating condition is retrieved and substituted into the adaptive Kalman filter.

[0024] In an optional implementation, the sample signal is decomposed into multi-scale frequency domain features using wavelet transform and Fourier transform, and the multi-scale frequency domain features are convolved to obtain multi-scale time-frequency features, including:

[0025]

[0026] in, It is the output fused frequency domain feature; It is frequency; It is a scale used to adjust the scaling of the wavelet function; is the translation, used to control the position of the wavelet function on the time axis; N is the number of samples, indicating the total number of time-domain samples; It is a sample of the state matrix output by the adaptive Kalman filter in the time domain, representing the signal value at discrete time n; It is the wavelet mother function; j is the imaginary unit;

[0027] By using a pre-trained convolutional neural network model to perform deep learning feature fusion on the fused frequency domain features, multi-scale time-frequency features are obtained.

[0028] In an optional implementation, the LSTM and CNN-based multi-task learning model includes:

[0029] The shared layer is a fully connected layer with an input size equal to the dimension of the data features and an output size equal to the specified hidden layer size. A ReLU activation function is added after the fully connected layer.

[0030] An LSTM layer is used to construct an LSTM network. The input size is the same as the output size of the shared layer, and the hidden layer size is the specified LSTM hidden layer size.

[0031] The CNN layer is constructed as a one-dimensional convolutional layer with the same number of input channels as the output size of the shared layer and the same number of output channels as the specified number of CNN channels. The kernel size is 3 and the padding is 1. A ReLU activation function is added after the convolutional layer.

[0032] Define a task-specific output layer to output the results of multiple tasks;

[0033] The Remaining Lifetime Prediction (RUL) layer is constructed as a fully connected layer with an input size equal to the concatenation of the outputs of the LSTM and CNN layers, and an output size of 1.

[0034] The fault type classification layer is constructed by building a fully connected layer. The input size is the size of the concatenated outputs of the LSTM and CNN layers, and the output size is the number of fault type categories. A Softmax activation function is added after the fully connected layer to output the probability distribution of fault types.

[0035] The health status assessment layer is constructed as a fully connected layer. The input size is the concatenation size of the outputs of the LSTM and CNN layers, and the output size is the number of health status categories. A Softmax activation function is added after the fully connected layer to output the probability distribution of health status.

[0036] The combined model combines shared layers, LSTM layers, CNN layers, and task-specific output layers into a multi-task model;

[0037] During forward propagation, the data first passes through a shared layer, and then is input into the LSTM layer and the CNN layer respectively. The outputs of the two layers are concatenated and then input into the task-specific output layer to obtain the prediction result for each task.

[0038] In an optional implementation, the training method for the multi-task learning model based on LSTM and CNN includes:

[0039] Set the total number of training epochs and the weights for the multi-task loss;

[0040] For each round of training:

[0041] Obtain a batch of input data, actual remaining lifetime, fault types, and health status from the training data loader;

[0042] Input data is fed into the model to obtain the prediction results for each task;

[0043] Calculate the loss for each task separately and add it to the loss list;

[0044] Calculate the total loss based on the multi-task loss function;

[0045] Update model parameters using the optimizer, including:

[0046] For the loss of each task, calculate its gradient separately;

[0047] Calculate the average gradient of all tasks as the reference gradient g0;

[0048] Let K be the number of tasks. This is the parameter vector of the model.

[0049] For the i-th task, its loss function is: The corresponding gradient is The update vector to be calculated is d;

[0050] The problem of calculating the update vector d can be formulated as a constrained optimization problem: in, It is a hyperparameter used to control the angular constraint between the update vector d and the reference gradient;

[0051] Construct the Lagrangian function to solve the constrained optimization problem:

[0052] in, They are Lagrange multipliers;

[0053] Since the constrained optimization problem is concave with respect to d, and when When the Slater condition is true, swapping the maximum and minimum operators yields the dual problem: in, w lies on the probability simplex, i.e. and ;

[0054] In a given In the case of w, the optimal update vector can be obtained by differentiating the Lagrange function with respect to d and setting it to zero. The expression: in, It is an M×M identity matrix;

[0055] calculate :

[0056] set up ;

[0057] First calculate ;

[0058] Recalculate The calculation result is denoted as D;

[0059] Finally, according to the SMW formula

[0060] calculate ;

[0061] make Then, by optimizing w in the dual problem, the optimal update vector is obtained, and the dual problem simplifies to: The simplified dual problem can be solved using methods such as gradient descent to obtain the optimal w;

[0062] Substitute the optimal w into Then g w Substitution The final update vector d is calculated. * Update vector d * It is a new gradient calculated by combining the reference gradient and the current task gradient;

[0063] The new gradient is then assigned back to the model's parameters.

[0064] Update model parameters using the optimizer.

[0065] Secondly, the present invention provides a port equipment operation and maintenance status identification system, comprising:

[0066] The acquisition module is used to acquire multi-source detection signals;

[0067] A denoising module is used to denoise the multi-source detection signal using an improved adaptive Kalman filter to obtain a sample signal;

[0068] The processing module is used to decompose the sample signal into multi-scale frequency domain features through wavelet transform and Fourier transform, and to perform convolution processing on the multi-scale frequency domain features to obtain multi-scale time-frequency features.

[0069] The identification module is used to identify the remaining lifetime, fault type, and health status corresponding to the multi-scale time-frequency features using a multi-task learning model based on LSTM and CNN.

[0070] Thirdly, a terminal is provided, including:

[0071] Memory, used to store port equipment operation and maintenance status identification programs;

[0072] A processor is configured to implement the steps of the port equipment operation and maintenance performance identification method as provided in the first aspect when executing the port equipment operation and maintenance performance identification program.

[0073] Fourthly, a computer-readable storage medium is provided, on which a port equipment operation and maintenance status identification program is stored, wherein when the port equipment operation and maintenance status identification program is executed by a processor, the port equipment operation and maintenance status identification method as provided in the first aspect is implemented.

[0074] The beneficial effects of this invention are as follows: The port equipment operation and maintenance status identification method, system, terminal, and medium provided by this invention, after obtaining multi-source detection signals, use an adaptive Kalman filter to select the appropriate filter according to the actual operating conditions of the port equipment to perform denoising, thereby achieving effective denoising of signals in dynamic scenes. By combining wavelet transform and Fourier transform with a convolutional neural network to extract multi-scale time-frequency features from multi-source signals, effective information in the detection signals can be fully extracted, and some noise can be further filtered out. The remaining lifetime, fault type, and health status corresponding to the multi-scale time-frequency features are identified using a multi-task learning model based on LSTM and CNN, achieving multi-task concurrent processing without redundant computation, thus improving monitoring efficiency and real-time performance.

[0075] Furthermore, the design principle of this invention is reliable, the structure is simple, and it has a very wide range of application prospects. Attached Figure Description

[0076] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0077] Figure 1 This is a schematic flowchart of a method according to an embodiment of the present invention.

[0078] Figure 2 This is an application flowchart of a method according to an embodiment of the present invention.

[0079] Figure 3 This is a schematic block diagram of a system according to an embodiment of the present invention.

[0080] Figure 4 This is a schematic diagram of the structure of a terminal provided in an embodiment of the present invention. Detailed Implementation

[0081] To enable those skilled in the art to better understand the technical solutions of this invention, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this invention.

[0082] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention.

[0083] The key terms used in this invention will be explained below.

[0084] A Convolutional Neural Network (CNN) is a deep learning model specifically designed to process data with a grid structure.

[0085] Long Short-Term Memory (LSTM) networks are a special type of recurrent neural network (RNN) that excels at processing sequential data.

[0086] The port equipment operation and maintenance status identification method provided in this embodiment of the invention is executed by a computer terminal, and correspondingly, the port equipment operation and maintenance status identification system runs on the computer terminal.

[0087] Figure 1 This is a schematic flowchart illustrating a method according to an embodiment of the present invention. Wherein, Figure 1 The implementing entity can be a port equipment operation and maintenance status identification system. Depending on different needs, the order of the steps in this flowchart can be changed, and some steps can be omitted.

[0088] like Figure 1 As shown, the method includes:

[0089] S1. Acquire multi-source detection signals;

[0090] S2. The multi-source detection signal is denoised using an improved adaptive Kalman filter to obtain a sample signal;

[0091] S3. The sample signal is decomposed into multi-scale frequency domain features by wavelet transform and Fourier transform, and the multi-scale frequency domain features are convolved to obtain multi-scale time-frequency features;

[0092] S4. Use a multi-task learning model based on LSTM and CNN to identify the remaining lifetime, fault type and health status corresponding to the multi-scale time-frequency features.

[0093] In one embodiment of the present invention, based on step S1, the following will provide a possible embodiment and describe its specific implementation in a non-limiting manner.

[0094] In the construction of a port equipment condition monitoring system, acquiring detection signals from a combination of sensors pre-deployed at multiple detection points on the port equipment is a crucial step in achieving accurate equipment condition assessment. These sensor combinations encompass temperature sensors, vibration sensors, current sensors, voltage sensors, pressure sensors, and displacement sensors.

[0095] Firstly, regarding hardware connectivity, various sensors are precisely installed at specific detection points based on the different structures and monitoring priorities of the port equipment. For example, temperature sensors are tightly attached to the surface of key heat-generating components of the equipment using dedicated thermally conductive patches or mounting brackets to ensure accurate sensing of temperature changes. Vibration sensors are securely installed on easily vibrating parts such as the equipment housing or bearing seats using magnetic or bolt fixing methods, ensuring a rigid connection with the equipment structure and effectively capturing vibration signals. Meanwhile, current sensors employ a non-invasive, open-loop structure, fitting onto the equipment's power supply line and acquiring current signals through electromagnetic induction. Voltage sensors are connected in parallel to the equipment's power supply circuit via dedicated voltage probes to achieve real-time acquisition of voltage signals. Pressure sensors are installed at specific interfaces on pipelines or pressure vessels according to the equipment's internal pressure monitoring requirements, accurately sensing pressure changes using pressure transmission media. Displacement sensors are connected to movable parts of the equipment via mechanical connections to monitor the displacement of these parts in real time.

[0096] Secondly, during signal transmission, all analog signals acquired by the sensors are transmitted to the signal conditioning module via dedicated signal transmission cables. These cables have excellent shielding performance to reduce the impact of external electromagnetic interference on the signal. The signal conditioning module performs preprocessing operations such as amplification, filtering, and analog-to-digital conversion on the incoming analog signals, converting them into digital signals suitable for subsequent processing. Subsequently, the conditioned digital signals are transmitted to the data acquisition server via wired networks (such as Ethernet) or wireless networks (such as Wi-Fi, LoRa, etc.). During transmission, reliable communication protocols are employed to ensure the accuracy and integrity of data transmission.

[0097] Finally, during the data acquisition phase, the data acquisition server uses specially developed data acquisition software to collect and store the transmitted sensor signals in real time according to the set sampling frequency and acquisition strategy.

[0098] For each sensor on the same port equipment, the original detection signal is segmented at a fixed time interval of 0.25 seconds. Specifically, a dedicated timestamp identification program is set up to accurately record the start time of the sensor signal according to the system clock. Starting from the start time, a signal segmentation command is triggered every 0.25 seconds.

[0099] At the hardware level, leveraging the powerful real-time data processing capabilities of a high-speed data acquisition card, the continuous analog signal within 0.25 seconds is rapidly converted into a discrete digital signal sequence. Simultaneously, an efficient data caching mechanism ensures the integrity and accuracy of the signal data during the segmentation process, preventing data loss or errors.

[0100] After segmentation, the obtained detection signal sequences are stored in a unified file format (such as CSV). During storage, a unique identifier is added to each sequence, containing key information such as sensor number and acquisition time. This ensures accurate retrieval and analysis of each sequence later, resulting in detection signal sequences of equal length and laying a solid foundation for subsequent data processing and analysis.

[0101] By using a time window to extract sample sequences from the detection signal sequence, and filling the constructed empty matrix with sample sequences corresponding to multiple sensors of the same port equipment, the detection signal matrix of the port equipment is obtained. .

[0102] In one embodiment of the present invention, based on step S2, the following will provide a possible embodiment and describe its specific implementation in a non-limiting manner.

[0103] Adaptive Kalman filters include:

[0104] Adaptive process noise covariance :

[0105]

[0106] in, It is a smoothing factor. It is the state matrix of the sensor at time k. It is the variance of the current sensor state matrix;

[0107] Adaptive Measurement Noise Covariance :

[0108]

[0109] in, It is a smoothing factor. It is the detection signal matrix generated at time k. It is the variance of the current detection signal matrix.

[0110] The state matrix is ​​a derivative matrix of the detection signal matrix. Based on the original detection signal from the sensor, the detection signal matrix can be directly obtained, and then the state matrix is ​​calculated from the detection signal matrix. Taking a port crane as an example:

[0111] Angle sensor: measures the angle of the crane boom, and the measured value at time k is... The unit is degrees.

[0112] Weight sensor: measures the weight of the lifted cargo, and the measured value at time k is... The unit is tons.

[0113] Speed ​​sensor: measures the crane's operating speed; the measured value at time k is... The unit is meters per second.

[0114] Accelerometer sensor: measures the acceleration of the crane during operation; the measured value at time k is... The unit is meters per second².

[0115] Current sensor: measures the motor's operating current; the measured value at time k is... The unit is ampere.

[0116] State value Defined as a 3×2 matrix, it is calculated in the following way:

[0117]

[0118] Among them, a1, b1, c1, d1, e1, f1, a2, b2, c2, d2, e2, f2 are weighting coefficients, which are set according to the degree of influence of each parameter on different aspects of the crane's state, and the sum of the coefficients of each row and each column is 1.

[0119] The noise reduction process includes:

[0120] S201. Port equipment faces various operating conditions during actual operation, such as full-load operation, no-load operation, acceleration start-up, and deceleration stop. For these different operating conditions, technicians need to conduct detailed research and testing beforehand. Through extensive experimental data and a deep understanding of the equipment's operating principles, the characteristic ranges of various sensor detection signals under each operating condition are determined. For example, during full-load operation, the value range of the pressure sensor detection signal may be between [X1, X2], while the value range of the temperature sensor detection signal may be in the range of [Y1, Y2]. Simultaneously, combining signal processing theory and practical application experience, a corresponding smoothing factor value is set for each operating condition. The smoothing factor value affects the smoothness and response speed of signal processing; a value that is too large results in an overly smooth signal but may lose some details; a value that is too small, while retaining details, may not effectively remove noise interference. After repeated debugging and verification, the most suitable smoothing factor value for different operating conditions is determined, and these values, along with the corresponding operating condition information and detection signal value ranges, are recorded in detail and stored in the system's parameter database.

[0121] S202. Upon acquiring multi-source detection signals, the system automatically initiates the operating condition matching process. First, the real-time acquired sensor detection signals are compared with the value ranges of the corresponding detection signals for each operating condition stored in the database, according to sensor type. For example, the current temperature sensor detection signal is matched one by one with the value ranges of temperature sensors under various operating conditions to determine which range it falls into. Simultaneously, the same matching operation is performed on the detection signals of other sensors such as vibration sensors and current sensors. To improve the accuracy and reliability of the matching, a comprehensive judgment method is adopted, selecting the operating condition with the most matched sensor detection signals as the matching operating condition. This comprehensive judgment method effectively avoids misjudgments caused by single sensor failure or external interference.

[0122] S203. Once the current operating condition is determined, the system immediately retrieves the corresponding smoothing factor value from the parameter database. The adaptive Kalman filter, as a powerful signal processing tool, can adjust the filtering parameters in real time according to the statistical characteristics of the signal, thereby achieving optimal signal estimation. Substituting the retrieved smoothing factor value into the adaptive Kalman filter, it will adjust the filter's prediction and update processes accordingly.

[0123] In adaptive Kalman filters, the prediction process and the update process are two key steps:

[0124] The prediction process includes:

[0125] State prediction: Based on the system's state transition model (a mathematical model describing how the system transitions from the current state to the next state), the state at the current moment is predicted using the state estimate obtained at the previous moment. For example, in the state monitoring of port cranes, if the state information such as the angle and speed of the crane boom at the previous moment is known, combined with the physical model of the crane's motion (state transition model), the estimated values ​​of the crane boom's angle, speed, and other states at the current moment can be predicted.

[0126] Covariance prediction: Simultaneously, the covariance of the state estimate is also predicted. Covariance reflects the degree of uncertainty in the state estimate. The calculation formula is as follows: ,in, It is the covariance of the state estimate from the previous time step. yes The transpose of the matrix, It is the process noise covariance of the previous moment.

[0127] It is the state transition matrix:

[0128]

[0129] It is the state estimate obtained in the previous moment. It is the state estimate at the current moment.

[0130] This calculation yields a measure of the uncertainty in the current state estimate, which is used for weight calculations when fusing it with measurements. In the adaptive Kalman filter, the smoothing factor... It will affect the process noise covariance The calculation affects the accuracy of state prediction, making the prediction more adaptable to changes in the system state under actual operating conditions.

[0131] The update process includes:

[0132] Measurement fusion: When the sensor measurement value at the current time k is obtained... Then, it is compared with the state prediction value obtained from the prediction process. The fusion is performed by calculating the Kalman gain. To determine the weights of measured and predicted values ​​when updating the state estimate. In an adaptive Kalman filter, the smoothing factor... It will affect the measurement noise covariance The calculation of this affects the Kalman gain. This ultimately affects the weighting of measured and predicted values.

[0133] State Update: The state estimate is updated based on the Kalman gain to obtain a more accurate state estimate for the current time step. By weighting the values ​​using Kalman gain and then adding them to the state prediction, a state estimate that better reflects the actual situation is obtained.

[0134] Covariance Update: Finally, the covariance of the state estimate. It is also updated to reflect the uncertainty of the updated state estimate. This updated covariance will be used in the prediction process for the next time step, forming a cyclical iterative filtering process.

[0135] Smoothing factor in the prediction process It participates in predicting the signal state at the next moment. During the prediction process, the adaptive Kalman filter bases its prediction on the system state equation and the current process noise covariance. The state at the next moment is estimated by combining a smoothing factor. The smoothing factor affects the accuracy of the prediction, making it more in line with the state changes under actual working conditions.

[0136] Smoothing factor during the update process It plays a role in the update process, influencing the weighting between measured and predicted values. The filter is based on the measurement noise covariance. and the acquired detection signal matrix By combining a smoothing factor to adjust the degree of confidence in the measured and predicted values, the two are merged to remove noise interference in the signal, obtain a more accurate state estimate, and output an effective signal that reflects the actual operating state of the equipment.

[0137] In one embodiment of the present invention, based on step S3, the following will provide a possible embodiment and describe its specific implementation in a non-limiting manner.

[0138] Existing feature extraction methods based on Fourier transform and wavelet transform are mostly limited to single-scale signal analysis and lack the integration of multi-scale information. In order to better diagnose equipment faults, a fusion method based on multi-scale analysis can be designed to combine Fourier transform, wavelet transform and deep learning methods to perform multi-level, multi-scale time-frequency feature extraction and fault diagnosis.

[0139] This application decomposes the output data of an adaptive Kalman filter into different scales using wavelet transform, and then performs Fourier transform on each scale to extract features within different frequency ranges. This allows for the capture of the equipment's operating characteristics at different scales and frequencies, especially abnormal patterns in vibration and noise. The frequency domain features obtained from wavelet transform and Fourier transform are then fused with deep learning features using a convolutional neural network (CNN) to obtain multi-scale time-frequency features, which are then applied to equipment fault classification and diagnosis.

[0140] Combination of wavelet transform and Fourier transform:

[0141]

[0142] in, It is the output fused frequency domain feature. It's frequency. It is a scale. It is a translation. It is the sample size. It is a sample of the state matrix output by the adaptive Kalman filter in the time domain. It is the wavelet mother function.

[0143] Specifically, the state matrix output by the adaptive Kalman filter is processed along the time dimension. Assuming the state matrix is ​​a two-dimensional matrix, each row represents the state at a given time point, and each column represents a different state variable (e.g., boom angle in one column, speed in another). Then, the elements of each column can be extracted sequentially according to their chronological order to form multiple time series.

[0144] For each time series, determine the sample size n. Starting from the first data point in the time series, number them sequentially as n = 0, 1, 2, ..., N, where N is the total number of samples in that time series. Thus, each data point in the time series can be represented by x(n), where n is the sample number.

[0145] When the state matrix contains multiple state variables (multiple columns), the time series x(n) corresponding to each state variable is integrated. In subsequent calculations combining wavelet transform and Fourier transform, each time series can be calculated separately, or they can be combined into a new sequence form as x(n) and input into the formula for calculation, depending on specific needs.

[0146] For example, if the state matrix has three columns, corresponding to the three state variables of angle, velocity, and weight, then three time series can be obtained. , , During computation, they are combined into a new x(n) by calculating a weighted sum and using computational rules.

[0147] This method allows for more accurate extraction of equipment vibration and noise characteristics from time-frequency signals, especially under complex operating conditions, which helps to identify subtle fault signs in a timely manner.

[0148] Methods for feature fusion using convolutional neural networks include:

[0149] Building a CNN Model: Construct a convolutional neural network model suitable for processing time-frequency features. The model typically contains multiple convolutional layers, pooling layers, and fully connected layers. Convolutional layers use convolutional kernels to slide across the time-frequency feature map, extracting local features; pooling layers downsample the feature map, reducing the amount of data while retaining key features. For example, in convolutional layers, choose an appropriate kernel size, such as 3×3 or 5×5, and set the number of kernels according to different feature extraction needs, generally starting with 32 or 64 and gradually increasing to learn richer features. Pooling layers can use max pooling or average pooling; a common pooling window size is 2×2.

[0150] Input time-frequency features: The frequency domain features obtained from wavelet transform and Fourier transform are organized into a format suitable for CNN input, usually in the form of a multi-dimensional tensor. For example, features at different scales and frequencies are arranged into a two-dimensional matrix, and then expanded into a three-dimensional tensor, which is used as input data for CNN. The first dimension represents the number of samples, the second dimension represents the feature scale, and the third dimension represents the frequency range.

[0151] Feature fusion process: During the forward propagation of a CNN, the input time-frequency features pass through convolutional layers and pooling layers sequentially. The convolutional layers fuse features of different scales and frequencies through convolution operations, learning more abstract and representative multi-scale time-frequency features. For example, after the first convolutional layer, features of different frequencies and scales are initially combined to form new feature maps, which contain comprehensive information about the device at different scales and frequencies. As the network deepens, features are continuously fused and refined, gradually forming more valuable feature representations for device fault classification and diagnosis.

[0152] In one embodiment of the present invention, based on step S4, the following will provide a possible embodiment and describe its specific implementation in a non-limiting manner.

[0153] Multi-task learning models based on LSTM and CNN include:

[0154] The shared layer is a fully connected layer with an input size equal to the dimension of the data features and an output size equal to the specified hidden layer size. A ReLU activation function is added after the fully connected layer.

[0155] An LSTM layer is used to construct an LSTM network. The input size is the same as the output size of the shared layer, and the hidden layer size is the specified LSTM hidden layer size.

[0156] The CNN layer is constructed as a one-dimensional convolutional layer with the same number of input channels as the output size of the shared layer and the same number of output channels as the specified number of CNN channels. The kernel size is 3 and the padding is 1. A ReLU activation function is added after the convolutional layer.

[0157] Define a task-specific output layer to output the results of multiple tasks;

[0158] The Remaining Lifetime Prediction (RUL) layer is constructed as a fully connected layer with an input size equal to the concatenation of the outputs of the LSTM and CNN layers, and an output size of 1.

[0159] The fault type classification layer is constructed by building a fully connected layer. The input size is the size of the concatenated outputs of the LSTM and CNN layers, and the output size is the number of fault type categories. A Softmax activation function is added after the fully connected layer to output the probability distribution of fault types.

[0160] The health status assessment layer is constructed as a fully connected layer. The input size is the concatenation size of the outputs of the LSTM and CNN layers, and the output size is the number of health status categories. A Softmax activation function is added after the fully connected layer to output the probability distribution of health status.

[0161] The combined model combines shared layers, LSTM layers, CNN layers, and task-specific output layers into a multi-task model;

[0162] During forward propagation, the data first passes through a shared layer, and then is input into the LSTM layer and the CNN layer respectively. The outputs of the two layers are concatenated and then input into the task-specific output layer to obtain the prediction result for each task.

[0163] Because gradient conflicts can easily occur between different tasks during the training of multi-task learning models, the training method for multi-task learning models based on LSTM and CNN provided in this application includes:

[0164] Set the total number of training epochs and the weights for the multi-task loss;

[0165] For each round of training:

[0166] Obtain a batch of input data, actual remaining lifetime, fault types, and health status from the training data loader;

[0167] Input data is fed into the model to obtain the prediction results for each task;

[0168] Calculate the loss for each task separately and add it to the loss list;

[0169] Calculate the total loss based on the multi-task loss function;

[0170] Update model parameters using the optimizer, including:

[0171] For the loss of each task, calculate its gradient separately;

[0172] Calculate the average gradient of all tasks as the reference gradient g0;

[0173] Let K be the number of tasks. This is the parameter vector of the model.

[0174] For the i-th task, its loss function is: The corresponding gradient is The update vector to be calculated is d;

[0175] The problem of calculating the update vector d can be formulated as a constrained optimization problem: in, It is a hyperparameter used to control the angular constraint between the update vector d and the reference gradient;

[0176] Construct the Lagrangian function to solve the constrained optimization problem:

[0177] in, They are Lagrange multipliers;

[0178] Since the constrained optimization problem is concave with respect to d, and when When the Slater condition is true, swapping the maximum and minimum operators yields the dual problem: in, w lies on the probability simplex, i.e. and ;

[0179] In a given In the case of w, the optimal update vector can be obtained by differentiating the Lagrange function with respect to d and setting it to zero. The expression: in, It is an M×M identity matrix;

[0180] calculate :

[0181] set up ;

[0182] First calculate ;

[0183] Recalculate The calculation result is denoted as D;

[0184] Finally, according to the SMW formula

[0185] calculate ;

[0186] make Then, by optimizing w in the dual problem, the optimal update vector is obtained, and the dual problem simplifies to: The simplified dual problem can be solved using methods such as gradient descent to obtain the optimal w;

[0187] Substitute the optimal w into Then g w Substitution The final update vector d is calculated. * Update vector d * It is a new gradient calculated by combining the reference gradient and the current task gradient;

[0188] The new gradients are then assigned back to the model's parameters so that the optimizer can update the parameters later.

[0189] The optimizer updates the model parameters. The optimizer's role is to update the model parameters based on the calculated gradients, thereby continuously reducing the value of the loss function. Common optimizers include stochastic gradient descent (SGD), Adam, and RMSProp. The optimizer adjusts the model parameters according to specific update rules (such as learning rate, momentum, etc.).

[0190] The trained multi-task learning model is used to identify the multi-scale time-frequency features extracted in step S3, obtaining remaining lifetime, fault type, and health status. The specific processing flow includes:

[0191] Shared layer processing: The preprocessed multi-scale time-frequency features are input into the shared layer, and after processing by a fully connected layer and the ReLU activation function, the output of the shared layer is obtained. The role of the shared layer is to extract general features of the data, providing a foundation for subsequent tasks.

[0192] LSTM and CNN layers process the data: The output of the shared layer is fed into the LSTM and CNN layers respectively. The LSTM layer processes the sequence data through its internal gating mechanism to capture temporal dependencies; the CNN layer extracts local features of the data through convolutional operations.

[0193] Feature concatenation: The outputs of the LSTM layer and the CNN layer are concatenated along the feature dimension to obtain a new feature vector that integrates time series information and local features.

[0194] Task-specific output layer processing: The concatenated feature vectors are input into the respective task-specific output layers.

[0195] RUL Prediction: The RUL layer performs a linear transformation on the input feature vector and outputs a numerical value representing the predicted remaining useful life of the device.

[0196] Fault type classification: After being processed by a fully connected layer and a Softmax activation function, the fault type classification layer outputs the probability distribution of each fault type. The category with the highest probability is the predicted fault type.

[0197] Health status assessment: The health status assessment layer is also processed by fully connected and softmax activation functions, and outputs the probability distribution of each health status category. The category with the highest probability is the predicted health status.

[0198] Please refer to Figure 2 In practical applications, the methods include:

[0199] 1. Hardware Configuration

[0200] Install multiple sensors (such as temperature, vibration, current, voltage, pressure, etc.) on key components to ensure comprehensive collection of equipment status data.

[0201] Sensor data undergoes initial processing by an edge computing unit before being transmitted wirelessly to a data center for analysis, reducing data transmission latency.

[0202] 2. Data Processing and Analysis System

[0203] Cloud platform: Stores and processes device data through a cloud computing platform, supporting large-scale data analysis.

[0204] Database: Use distributed databases (such as Hadoop and Spark) for device data storage and management.

[0205] 3. Model Training and Deployment

[0206] Training dataset: A large amount of equipment operation and failure data is collected for model training. LSTM and CNN models are built using deep learning frameworks.

[0207] Model deployment: Deploy the trained model to edge computing devices and cloud platforms to perform real-time device status identification and fault prediction.

[0208] 4. Data Processing

[0209] An adaptive Kalman filter is used to preprocess multi-source detection signals to remove noise. A multi-scale time-frequency feature-based fault diagnosis method combining wavelet transform-Fourier transform and CNN is used to extract multi-scale time-frequency features. A multi-task learning model is used to identify remaining service life, fault type, and health status from the multi-scale time-frequency features. Weighted fusion of sensor data is performed, and the output of the multi-task learning model is combined to generate operation and maintenance decisions.

[0210] In some embodiments, the port equipment operation and maintenance status identification system may include multiple functional modules composed of computer program segments. The computer programs for each program segment in the port equipment operation and maintenance status identification system may be stored in the memory of a computer terminal and executed by at least one processor to perform (see details). Figure 1 (Description) Function for identifying the operational status of port equipment.

[0211] In this embodiment, the port equipment operation and maintenance status identification system can be divided into multiple functional modules based on its functions, such as... Figure 3 As shown. The system's functional modules may include: an acquisition module, a noise reduction module, a processing module, and a recognition module. The module referred to in this invention is a series of computer program segments that can be executed by at least one processor and perform a fixed function, stored in memory. In this embodiment, the functions of each module will be detailed in subsequent embodiments.

[0212] The acquisition module is used to acquire multi-source detection signals;

[0213] A denoising module is used to denoise the multi-source detection signal using an improved adaptive Kalman filter to obtain a sample signal;

[0214] The processing module is used to decompose the sample signal into multi-scale frequency domain features through wavelet transform and Fourier transform, and to perform convolution processing on the multi-scale frequency domain features to obtain multi-scale time-frequency features.

[0215] The identification module is used to identify the remaining lifetime, fault type, and health status corresponding to the multi-scale time-frequency features using a multi-task learning model based on LSTM and CNN.

[0216] Figure 4The port equipment operation and maintenance status identification method provided in the embodiments of this application can be applied to terminals. Those skilled in the art will understand that the terminal structure involved in the embodiments of this invention does not constitute a limitation on the terminal. A terminal may include more or fewer components than illustrated, or combine certain components, or have different component arrangements. In the embodiments of this invention, the terminal includes, but is not limited to, laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. A terminal may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable terminals, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the embodiments of this application described and / or claimed herein.

[0217] The terminal 400 may include a processor 410, a memory 420, and a communication unit 430. These components communicate via one or more buses. Those skilled in the art will understand that the server structure shown in the figure does not constitute a limitation of the present invention. It may be a bus topology or a star topology, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0218] The memory 420 can be used to store the execution instructions of the processor 410. The memory 420 can be implemented by any type of volatile or non-volatile storage terminal or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. When the execution instructions in the memory 420 are executed by the processor 410, the terminal 400 is able to perform some or all of the steps in the above method embodiments.

[0219] The processor 410 serves as the control center of the storage terminal, connecting various parts of the electronic terminal via various interfaces and lines. It executes software programs and / or modules stored in the memory 420, and calls data stored in the memory to perform various functions of the electronic terminal and / or process data. The processor can be composed of integrated circuits (ICs), such as a single packaged IC or multiple packaged ICs with the same or different functions connected together. For example, the processor 410 may consist only of a central processing unit (CPU). In this embodiment of the invention, the CPU may have a single processing core or include multiple processing cores.

[0220] The communication unit 430 is used to establish a communication channel, enabling the storage terminal to communicate with other terminals. It can receive user data sent by other terminals or send user data to other terminals.

[0221] The present invention also provides a computer storage medium, wherein the computer storage medium may store a program, which, when executed, may include some or all of the steps provided in the embodiments of the present invention. The storage medium may be a magnetic disk, an optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0222] Those skilled in the art will clearly understand that the techniques in the embodiments of the present invention can be implemented using software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solutions in the embodiments of the present invention, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium such as a USB flash drive, mobile hard drive, read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk, or other media capable of storing program code. It includes several instructions to cause a computer terminal (which may be a personal computer, server, or a second terminal, network terminal, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention.

[0223] The same or similar parts between the various embodiments in this specification can be referred to mutually. In particular, the terminal embodiments are basically similar to the method embodiments, so the description is relatively simple, and the relevant parts can be referred to the description in the method embodiments.

[0224] In the embodiments provided by this invention, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between systems or modules may be electrical, mechanical, or other forms.

[0225] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0226] In addition, the functional modules in the various embodiments of the present invention can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module.

[0227] Although the present invention has been described in detail with reference to the accompanying drawings and preferred embodiments, the present invention is not limited thereto. Various equivalent modifications or substitutions can be made to the embodiments of the present invention by those skilled in the art without departing from the spirit and essence of the invention, and such modifications or substitutions should all be within the scope of the present invention. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should also be covered within the protection scope of the present invention.

Claims

1. A method for identifying the operational status of port equipment, characterized in that, include: Acquire multi-source detection signals; The multi-source detection signal is denoised using an improved adaptive Kalman filter to obtain a sample signal; The sample signal is decomposed into multi-scale frequency domain features by wavelet transform and Fourier transform, and the multi-scale frequency domain features are convolved to obtain multi-scale time-frequency features. The remaining lifetime, fault type, and health status corresponding to the multi-scale time-frequency features are identified using a multi-task learning model based on LSTM and CNN. The adaptive Kalman filter includes: Adaptive process noise covariance : in, It is a smoothing factor. It is the state matrix of the sensor at time k. It is the variance of the current sensor state matrix; Adaptive Measurement Noise Covariance : in, It is a smoothing factor. It is the detection signal matrix generated at time k. It is the variance of the current detection signal matrix; The multi-source detection signal is denoised using an improved adaptive Kalman filter to obtain a sample signal, including: Pre-set smoothing factor values ​​for various operating conditions, as well as the range of detection signal values; The multi-source detection signal is matched with the detection signal value range corresponding to each working condition to determine the current working condition; Retrieve the smoothing factor value corresponding to the current operating condition and substitute it into the adaptive Kalman filter; The sample signal is decomposed into multi-scale frequency domain features using wavelet transform and Fourier transform, and then convolutional processing is performed on the multi-scale frequency domain features to obtain multi-scale time-frequency features, including: in, It is the output fused frequency domain feature; It is frequency; It is a scale used to adjust the scaling of the wavelet function; is the translation, used to control the position of the wavelet function on the time axis; N is the number of samples, indicating the total number of time-domain samples; It is a sample of the state matrix output by the adaptive Kalman filter in the time domain; It is the wavelet mother function; j is the imaginary unit; By using a pre-trained convolutional neural network model to perform deep learning feature fusion on the fused frequency domain features, multi-scale time-frequency features are obtained. The fused frequency domain features obtained from wavelet transform and Fourier transform are organized into a format suitable for CNN input. The features at different scales and frequencies are arranged into a two-dimensional matrix, which is then expanded into a three-dimensional tensor as the input data for CNN. The first dimension represents the number of samples, the second dimension represents the feature scale, and the third dimension represents the frequency range.

2. The method according to claim 1, characterized in that, Acquire multi-source detection signals, including: The detection signals of a sensor combination pre-deployed at multiple detection points in port equipment are acquired. The sensor combination includes a temperature sensor, a vibration sensor, a current sensor, a voltage sensor, a pressure sensor, and a displacement sensor.

3. The method according to claim 1, characterized in that, The multi-task learning model based on LSTM and CNN includes: The shared layer is a fully connected layer with an input size equal to the dimension of the data features and an output size equal to the specified hidden layer size. A ReLU activation function is added after the fully connected layer. An LSTM layer is used to construct an LSTM network. The input size is the same as the output size of the shared layer, and the hidden layer size is the specified LSTM hidden layer size. The CNN layer is constructed as a one-dimensional convolutional layer with the same number of input channels as the output size of the shared layer and the same number of output channels as the specified number of CNN channels. The kernel size is 3 and the padding is 1. A ReLU activation function is added after the convolutional layer. Define the output layer for the task, which is used to output the results of multiple tasks; The remaining lifespan prediction layer is constructed as a fully connected layer with an input size equal to the concatenation of the outputs of the LSTM and CNN layers, and an output size of 1. The fault type classification layer is constructed by building a fully connected layer. The input size is the size of the concatenated outputs of the LSTM and CNN layers, and the output size is the number of fault type categories. A Softmax activation function is added after the fully connected layer to output the probability distribution of fault types. The health status assessment layer is constructed as a fully connected layer. The input size is the concatenation size of the outputs of the LSTM and CNN layers, and the output size is the number of health status categories. A Softmax activation function is added after the fully connected layer to output the probability distribution of health status. The combined model combines shared layers, LSTM layers, CNN layers, and the task's output layer into a multi-task model; During forward propagation, the data first passes through the shared layer, and then is input into the LSTM layer and CNN layer respectively. The outputs of the two layers are concatenated and then input into the output layer of each task to obtain the prediction result for each task.

4. The method according to claim 3, characterized in that, Training methods for multi-task learning models based on LSTM and CNN include: Set the total number of training epochs and the weights for the multi-task loss; For each round of training: Obtain a batch of input data, actual remaining lifetime, fault types, and health status from the training data loader; Input data is fed into the model to obtain the prediction results for each task; Calculate the loss for each task separately and add it to the loss list; Calculate the total loss based on the multi-task loss function; Update model parameters using the optimizer, including: For the loss of each task, calculate its gradient separately; Calculate the average gradient of all tasks as the reference gradient. ; Let K be the number of tasks. This is the parameter vector of the model; For the i-th task, its loss function is: The corresponding gradient is The update vector to be calculated is d; The problem of calculating the update vector d can be formulated as a constrained optimization problem: in, It is a hyperparameter used to control the angular constraint between the update vector d and the reference gradient; Construct the Lagrangian function to solve the constrained optimization problem: in, They are Lagrange multipliers; Since the constrained optimization problem is concave with respect to d, and when When the Slater condition is true, swapping the maximum and minimum operators yields the dual problem: in, w lies on the probability simplex, i.e. and ; In a given In the case of w, the optimal update vector can be obtained by differentiating the Lagrange function with respect to d and setting it to zero. The expression: in, It is an M×M identity matrix; calculate : set up ; First calculate ; Recalculate The calculation result is denoted as D; Finally, according to the SMW formula calculate ; make Then, by optimizing w in the dual problem, the optimal update vector is obtained, and the dual problem simplifies to: The gradient descent method is used to solve the simplified dual problem to obtain the optimal w; Substitute the optimal w into And then Substitution The final update vector d* is calculated, which is the new gradient calculated by combining the reference gradient and the current task gradient; The new gradient is then assigned back to the model's parameters. Update model parameters using the optimizer.

5. A port equipment operation and maintenance status identification system, characterized in that, include: The acquisition module is used to acquire multi-source detection signals; A denoising module is used to denoise the multi-source detection signal using an improved adaptive Kalman filter to obtain a sample signal; The processing module is used to decompose the sample signal into multi-scale frequency domain features through wavelet transform and Fourier transform, and to perform convolution processing on the multi-scale frequency domain features to obtain multi-scale time-frequency features. The identification module is used to identify the remaining lifetime, fault type and health status corresponding to the multi-scale time-frequency features using a multi-task learning model based on LSTM and CNN. The adaptive Kalman filter includes: Adaptive process noise covariance : in, It is a smoothing factor. It is the state matrix of the sensor at time k. It is the variance of the current sensor state matrix; Adaptive Measurement Noise Covariance : in, It is a smoothing factor. It is the detection signal matrix generated at time k. It is the variance of the current detection signal matrix; The multi-source detection signal is denoised using an improved adaptive Kalman filter to obtain a sample signal, including: Pre-set smoothing factor values ​​for various operating conditions, as well as the range of detection signal values; The multi-source detection signal is matched with the detection signal value range corresponding to each working condition to determine the current working condition; Retrieve the smoothing factor value corresponding to the current operating condition and substitute it into the adaptive Kalman filter; The sample signal is decomposed into multi-scale frequency domain features using wavelet transform and Fourier transform, and then convolutional processing is performed on the multi-scale frequency domain features to obtain multi-scale time-frequency features, including: in, It is the output fused frequency domain feature; It is frequency; It is a scale used to adjust the scaling of the wavelet function; is the translation, used to control the position of the wavelet function on the time axis; N is the number of samples, indicating the total number of time-domain samples; It is a sample of the state matrix output by the adaptive Kalman filter in the time domain; It is the wavelet mother function; j is the imaginary unit; By using a pre-trained convolutional neural network model to perform deep learning feature fusion on the fused frequency domain features, multi-scale time-frequency features are obtained. The fused frequency domain features obtained from wavelet transform and Fourier transform are organized into a format suitable for CNN input. The features at different scales and frequencies are arranged into a two-dimensional matrix, which is then expanded into a three-dimensional tensor as the input data for CNN. The first dimension represents the number of samples, the second dimension represents the feature scale, and the third dimension represents the frequency range.

6. A terminal, characterized in that, include: Memory, used to store port equipment operation and maintenance status identification programs; A processor is used to implement the steps of the port equipment operation and maintenance performance identification method according to any one of claims 1-4 when executing the port equipment operation and maintenance performance identification program.

7. A computer-readable storage medium storing a computer program, characterized in that, The readable storage medium stores a port equipment operation and maintenance status identification program, which, when executed by a processor, implements the steps of the port equipment operation and maintenance status identification method according to any one of claims 1-4.

Images