Valve opening and closing state recognition and diagnosis method based on deep learning

Abstract

The invention discloses a deep learning-based valve opening and closing state recognition and diagnosis method, which comprises the following steps of: acquiring and synchronizing multi-source signals, and generating a standardized multi-mode time sequence sample; extracting modal features by multiple branches and fusing the modal features into a joint feature vector sequence; the combined features are input into a phase change layered decoder, and a layered recognition result is output; constructing a plurality of types of abnormal events in a point process layer modeling phase change stage, and outputting an event modeling result; constructing a condition reversible generation diagnostor, and outputting a consistency checking result; and fusing a result output state and diagnosis information, and executing alarming and filing. Through multi-mode deep learning feature fusion, phase change hierarchical decoding and conditional modeling, accurate recognition of the opening and closing state of the valve, fine division of the phase change stage and intelligent diagnosis of early faults are achieved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of industrial equipment state monitoring, and in particular to a valve opening and closing state recognition and diagnosis method based on deep learning. BACKGROUND

[0002] In industrial automation and process control systems, valves, as key execution components, are widely used in petrochemical, power, metallurgy, water conservancy and other fields. The accurate recognition and fault diagnosis of the opening and closing state of the valve is directly related to the safety and stability of the production process. In the prior art, the recognition method of the opening and closing state of the valve depends on a single sensor signal, such as current, voltage or pressure signal, and the state is distinguished by setting a threshold or pattern matching. Although this method is simple to implement, it can only coarsely distinguish between open and closed states of the valve, and cannot finely capture the dynamic characteristics of the phase change stage in the opening and closing process, especially in complex working conditions, which is prone to misjudgment and omission. The single signal is sensitive to environmental noise and equipment aging, and the diagnosis result lacks robustness and credibility.

[0003] With the development of deep learning, some studies have begun to try to use convolutional neural networks, recurrent neural networks and other models to analyze sensor signals, so as to realize intelligent recognition of the valve state. However, this method usually only processes single-mode data, and fails to fully utilize the complementary information between multi-modal data such as acoustics, vibration, electricity and video, resulting in obvious deficiencies in capturing transient abnormal events (such as stuck micro-pulse, leakage sound cluster) in the phase change stage of the valve opening and closing. The prior art lacks a consistency checking mechanism between the recognition result and the input observation in the generation of the diagnosis process, making it difficult to trace and explain the diagnosis result, and difficult to meet the requirements of high confidence and explainability in industrial applications.

[0004] Therefore, the existing valve opening and closing state recognition technology has the following defects: first, the recognition granularity is insufficient, only the open and closed states can be judged, and there is a lack of fine recognition of the phase change stage; second, there is a lack of modeling ability for transient abnormal events, making it difficult to find early fault hidden dangers in time; third, there is a lack of multi-modal consistency checking mechanism, resulting in insufficient diagnosis accuracy and explainability.

[0005] Therefore, how to provide a valve opening and closing state recognition and diagnosis method based on deep learning is a problem that those skilled in the art need to solve. SUMMARY

[0006] One purpose of the present application is to propose a valve opening and closing state recognition and diagnosis method based on deep learning, which fully utilizes multi-modal deep feature extraction, phase change hierarchical decoding and neural point process modeling technology, and constructs a conditional reversible generation diagnostic device to realize comprehensive recognition and checking of valve opening and closing state, phase change stage, transient abnormal event and fault type. The present application describes in detail the overall process of multi-modal data acquisition and synchronization, feature extraction and cross-modal fusion, hierarchical decoder output opening and closing state and phase change stage, neural Hawkes point process modeling of transient events, and consistency checking of conditional reversible generation diagnostic device, which can realize fine identification of valve opening and closing state and accurate diagnosis of early faults under complex working conditions. The method has the advantages of high recognition accuracy, sensitivity to transient abnormal events, interpretable diagnosis results, strong robustness and adaptability to complex working conditions.

[0007] According to the valve opening and closing state recognition and diagnosis method based on deep learning of the embodiment of the present application, the method comprises: Acquire acoustic signals, vibration signals, electrical signals, video signals and actuator control quantity signals during valve operation, synchronize at the millisecond level based on a unified time reference, perform denoising, segmentation and normalization processing, perform time-frequency transformation on the acoustic signals and vibration signals to obtain time-frequency spectrograms, and perform amplitude and morphology standardization on the electrical signals and video signals to form multi-modal time series samples; Input the time-frequency spectrograms into a time-frequency convolution branch, input the electrical signals into a time series modeling branch, and input the video signals into a lightweight image branch to respectively extract branch features and generate a joint feature vector sequence through cross-modal interactive attention fusion; Input the joint feature vector sequence into a phase change hierarchical decoder, output hierarchical recognition results by level, determine a phase change stage time window and output an opening degree curve; In the phase change stage time window, construct a phase change conditional point process layer based on the joint feature vector sequence, model the arrival time and occurrence intensity of limit impact events, stuck micro-pulse events and leakage voiceprint cluster events using a neural Hawkes point process, and output event modeling results; Form a condition set with the phase change stage sequence and the opening degree curve, and the actuator control quantity signal, construct a conditional reversible generation diagnostic device, perform conditional generation and reversible reconstruction on the multi-modal time series samples, and output consistency checking results; Fuse the hierarchical recognition results, event modeling results and consistency checking results to obtain valve opening and closing state, opening degree, phase change stage, fault type and severity score, trigger an alarm and issue a linkage control instruction when the score exceeds a preset threshold, and perform drift monitoring and recalibration and archive the diagnosis results.

[0008] Optionally, the millisecond-level synchronization based on a unified time reference, and the denoising, segmentation and normalization processing, refer to aligning acoustic signals, vibration signals, electrical signals, video signals and actuator control signals according to the same time reference, using filtering to remove high-frequency noise and DC components, dividing the continuous signal into multiple time segments according to a preset sampling length, and unifying different modal signals to a comparable scale through amplitude standardization and mean-variance normalization.

[0009] Optionally, the step of performing time-frequency transformation on acoustic and vibration signals to obtain time-frequency spectrograms, and the step of amplitude and shape standardization on electrical and video signals, refers to obtaining frequency distribution characteristics and forming a two-dimensional time-frequency spectrogram by transforming acoustic and vibration signals from the time domain to the frequency domain within a fixed time window, eliminating amplitude differences between different measurement channels by normalizing the amplitude of electrical signals, and performing shape standardization on video signals by scaling and equalizing grayscale distribution.

[0010] Optionally, the step of generating a joint feature vector sequence through cross-modal interactive attention fusion includes: From the formed multimodal time-series samples, the time spectrum of acoustics and vibration, the time segments of electrical signals, and video frames or frame groups aligned with the time segments are obtained according to a unified time window. The time-frequency spectrum is input into the time-frequency convolution branch in channel order, and convolution processing and feature compression are performed sequentially to obtain a time-frequency branch feature vector sequence that corresponds one-to-one with the time window. The time segments of the electrical signal are input into the timing modeling branch, and timing modeling and dimension regularization are performed sequentially to obtain a sequence of timing branch feature vectors that correspond one-to-one with the time window. The aligned video frames or frame groups are input into the lightweight image branch, and convolution and downsampling are performed sequentially to obtain a sequence of image branch feature vectors corresponding to the time window. The feature vector sequences of the time-frequency branch, the time-series branch, and the image branch are concatenated in the channel dimension according to the time window order. The concatenated feature vector sequences are then weighted and fused based on cross-modal interactive attention to generate a joint feature vector sequence.

[0011] Optionally, the step of inputting the joint feature vector sequence into the phase transition hierarchical decoder, outputting hierarchical recognition results according to the hierarchy, determining the time window of the phase transition stage, and outputting the opening curve includes: A phase-change hierarchical decoder is constructed, which consists of a first-layer state decoding layer, a second-layer stage decoding layer and a third-layer fault decoding layer. The joint feature vector sequence is input into the phase-change hierarchical decoder. In the first state decoding layer, the joint feature vector sequence is decoded in chronological order to output the open and closed state sequence; In the second-level stage decoding layer, based on the preset phase sequence mask, only the information transmission between the start-up stage and the acceleration stage, the acceleration stage and the stabilization stage, the stabilization stage and the deceleration stage, and the deceleration stage and the limit contact stage is allowed. In addition, a monotonic gating rule is applied in combination with the start-up and stop state sequence and the actuator control signal to output the phase transition stage sequence and determine the phase transition stage time window. In the third fault decoding layer, context routing is performed between the open / closed state sequence and the phase transition stage sequence in the stage-specific fault path, and the initial fault judgment label sequence corresponding to each phase transition stage is output. The opening regression head built into the phase change hierarchical decoder outputs an opening curve aligned with the phase change stage time window under the common constraints of the opening / closing state sequence and the phase change stage sequence. The opening / closing state sequence, the phase change stage sequence, and the initial fault label sequence are collectively referred to as the hierarchical identification result.

[0012] Optionally, the opening regression head refers to a regression submodule set in the phase change layer decoder for outputting the valve opening change curve over time. The regression submodule takes the joint feature vector sequence, the opening and closing state sequence, the phase change stage sequence, and the actuator control signal as input, performs time alignment according to the phase change stage time window, applies monotonically non-decreasing gating in the opening stage, applies monotonically non-increasing gating in the closing stage, and performs boundary clamping in the limit contact stage, outputting an opening curve with a consistent sampling period and a value limited to the range of 0% to 100%.

[0013] Optionally, the output event modeling results include: The received phase transition stage sequence and phase transition stage time window, as well as the output joint feature vector sequence; For each phase transition stage time window, a feature subsequence aligned with the phase transition stage time window is extracted from the joint feature vector sequence to determine the event type as limit impact event, stuck micropulse event and leakage acoustic pattern cluster event. A phase transition condition point process layer is constructed based on feature subsequences. This layer includes a stage-specific triggering kernel library, event mutual inhibition and cooperative gating, cross-modal confidence weighting, and a multi-resolution temporal grid. The phase-specific triggering kernel libraries correspond to the startup phase, acceleration phase, stabilization phase, deceleration phase, and limit contact phase, respectively. Event mutual inhibition and collaborative gating constrain the triggering relationship of different event types within the same time window. They suppress unreasonable concurrent triggering through mutual exclusion mechanism and strengthen joint triggering with causal relationship through collaborative mechanism. Cross-modal confidence weighting weights event scores based on the feature confidence levels of each branch; Multi-resolution temporal raster simultaneously characterizes event triggering at both fine-grained and coarse-grained time scales; Within each phase transition stage time window, the arrival time and intensity of limit impact events, stuck micropulse events and leakage acoustic cluster events are modeled using neural Hawkes point processes, generating a candidate event timestamp set, outputting the event intensity curve that changes with time, and calculating the expected event count and the posterior distribution of event types. The execution phase boundary alignment and limit calibration involves matching the candidate event timestamp set with the boundary of the phase transition phase time window, eliminating candidate event timestamps that fall outside the boundary, and determining the actual event time at the boundary through the alignment calibration of the limit impact event. Valid trigger points are then selected by combining the confidence scores of stuck micropulse events and leakage acoustic cluster events to obtain the event timestamp set. The event timestamp set, event intensity curve, expected event count, and posterior distribution of event type for each phase transition stage time window are summarized into the event modeling results.

[0014] Optionally, the output consistency verification result includes: Receive phase transition phase sequence, phase transition phase time window, opening curve, actuator control signal and multimodal timing sample, and perform time alignment according to the phase transition phase time window; A conditional reversible generation diagnostic tool is constructed, which consists of a conditional encoding unit, a reversible mapping unit, and a consistency evaluation unit, wherein: The reversible mapping unit is composed of multiple reversible coupled mapping layers connected in series. Each level contains a scaling transformation sub-layer and a translation transformation sub-layer while maintaining the consistency of input and output dimensions. The consistency evaluation unit consists of a log-likelihood evaluation module and a residual aggregation module. The log-likelihood evaluation module outputs the conditional log-likelihood sequence for each time step, and the residual aggregation module outputs the modal-level reconstructed residual spectrum. Within each phase transition stage time window, the conditional encoding unit combines the phase transition stage sequence, the stage identifier corresponding to the phase transition stage time window, the opening curve, and the actuator control signal into a conditional vector sequence according to the time step, and completes length normalization and channel alignment. For each time step, the multimodal time series samples and the condition vector sequence are input into the invertible mapping unit, and forward mapping is performed to obtain the internal representation. Then, inverse mapping is performed under the same condition vector to obtain the reconstructed fragment, forming the reconstructed sequence corresponding to the time step. The consistency evaluation unit calculates the time-domain and frequency-domain residuals of the acoustic and vibration channels, the amplitude and morphological residuals of the electrical channels, and the spatial and temporal residuals of the video channels for each time step of the reconstructed sequence and the original sequence. These residuals are then summarized into a modal-level reconstruction residual spectrum. Simultaneously, the conditional log-likelihood sequence corresponding to each time step is output. The conditional log-likelihood sequence and the modal-level reconstruction residual spectrum are collectively referred to as the consistency verification result.

[0015] Optionally, the fusion of the hierarchical identification results, event modeling results, and consistency verification results yields valve opening / closing status, opening degree, phase transition stage, fault type, and severity score, including: The hierarchical identification results, event modeling results, and consistency verification results are time-aligned according to the phase transition stage time window under the same time base, and resampled according to a unified sampling period. The hierarchical identification confidence, event arrival rate and expected event count, log likelihood and modal residual are interval-standardized. Within each phase transition stage, based on a preset weight table and stage gating rules, the hierarchical identification results are mapped to state evidence, the event modeling results are mapped to event evidence, and the consistency verification results are mapped to consistency evidence. The three types of evidence are weighted and synthesized to generate stage-level fusion scores and modality-level fusion scores. The valve opening and closing status and phase transition stage are determined based on the stage-level fusion score and the modal-level fusion score, the fault type is determined, the opening degree is obtained by combining the opening curve, and the severity score is calculated.

[0016] The beneficial effects of this invention are: This invention introduces a multimodal deep learning feature extraction and cross-modal interactive fusion mechanism during valve opening and closing, enabling simultaneous analysis of multi-source data, including acoustic, vibration, electrical, and video data. Compared to existing methods that rely on a single signal threshold for determination, this invention can capture richer dynamic features of opening and closing and form a refined state division during the phase transition stage. This effectively solves the limitation of traditional technologies that can only coarsely distinguish between open and closed states, significantly improving the accuracy and stability of valve opening and closing state recognition.

[0017] This invention constructs an identification framework that combines a phase-change hierarchical decoder with a neural Hawkes point process. This framework not only outputs the opening and closing states, phase-change stages, and initial fault assessment results layer by layer, but also models transient abnormal events such as limit impacts, stuck micro-pulses, and leakage acoustic signature clusters within the phase-change stage time window. This innovative design enables the system to identify potential latent faults at an early stage, improving the sensitivity and detection rate of abnormal operating conditions and enhancing the foresight and timeliness of valve operation status monitoring.

[0018] The conditionally reversible generator diagnostic tool proposed in this invention reconstructs and verifies multimodal time-series samples under given conditions, outputting a log-likelihood sequence and modal-level residual spectrum. This effectively establishes consistency constraints between diagnostic results and original observations, improving the reliability and interpretability of diagnostic results. As a result, valve status identification and fault diagnosis are no longer just "black box" outputs, but have traceable and reliable evidence, providing higher safety and robustness for industrial automation operation under complex working conditions. Attached Figure Description

[0019] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart of a valve opening and closing state recognition and diagnosis method based on deep learning proposed in this invention; Figure 2 This is a schematic diagram of the three-layer decoding structure of a phase-change hierarchical decoder for a deep learning-based valve opening and closing state recognition and diagnosis method proposed in this invention. Detailed Implementation

[0020] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0021] refer to Figure 1 and Figure 2 A deep learning-based method for valve opening and closing status recognition and diagnosis includes: Acoustic signals, vibration signals, electrical signals, video signals, and actuator control signals are collected during valve operation. Based on a unified time reference, millisecond-level synchronization is performed, followed by denoising, segmentation, and normalization. Time-frequency transformation is performed on acoustic and vibration signals to obtain time-spectrum diagrams, and amplitude and morphology are standardized for electrical and video signals to form multimodal time-series samples. The time-frequency spectrum is input into the time-frequency convolution branch, the electrical signal is input into the time-series modeling branch, and the video signal is input into the lightweight image branch. The branch features are extracted respectively, and a joint feature vector sequence is generated by cross-modal interactive attention fusion. The joint feature vector sequence is input into the phase transition hierarchical decoder, and the hierarchical recognition results are output according to the hierarchy. The time window of the phase transition stage is determined and the opening curve is output. Within the phase transition phase time window, a phase transition condition point process layer is constructed based on the joint feature vector sequence. The arrival time and occurrence intensity of limit impact events, stuck micropulse events and leakage acoustic pattern cluster events are modeled using neural Hawkes point processes, and the event modeling results are output. Using the phase transition stage sequence, opening curve, and actuator control signal as a condition set, a condition reversible generation diagnostic tool is constructed to perform condition generation and reversible reconstruction on multimodal time series samples and output consistency verification results. The results of hierarchical identification, event modeling, and consistency verification are integrated to obtain the valve opening and closing status, opening degree, phase change stage, fault type, and severity score. When the preset threshold is exceeded, an alarm is triggered and a linkage control command is issued. At the same time, drift monitoring and recalibration are performed and the diagnostic results are archived.

[0022] In this embodiment, the millisecond-level synchronization based on a unified time reference, and the denoising, segmentation and normalization processing, refer to aligning acoustic signals, vibration signals, electrical signals, video signals and actuator control signals according to the same time reference, using filtering to remove high-frequency noise and DC components, dividing continuous signals into multiple time segments according to a preset sampling length, and unifying different modal signals to a comparable scale through amplitude standardization and mean-variance normalization.

[0023] In this embodiment, the time-frequency transformation of acoustic and vibration signals to obtain time-frequency spectrum diagrams, and the amplitude and shape standardization of electrical and video signals refer to obtaining frequency distribution characteristics and forming two-dimensional time-frequency spectrum diagrams by transforming acoustic and vibration signals from the time domain to the frequency domain within a fixed time window, eliminating amplitude differences between different measurement channels by normalizing the amplitude of electrical signals, and performing shape standardization processing on video signals by scaling and equalizing grayscale distribution.

[0024] In this embodiment, the step of generating a joint feature vector sequence through cross-modal interactive attention fusion includes: From the formed multimodal time-series samples, the time spectrum of acoustics and vibration, the time segments of electrical signals, and video frames or frame groups aligned with the time segments are obtained according to a unified time window. The time-frequency spectrogram is input into the time-frequency convolution branch in channel order, and convolution processing and feature compression are performed sequentially to obtain a time-frequency branch feature vector sequence corresponding to the time window. The sequential convolution processing and feature compression refer to using a two-dimensional convolution kernel to perform local weighted calculations along the time and frequency dimensions on the input time-frequency spectrogram, extracting local energy distribution features, and outputting an enhanced time-frequency pattern through a nonlinear activation function. Pooling and dimensionality reduction operations are performed on the feature map to remove redundant information and retain the main discriminative features. The time segment of the electrical signal is input into the time-series modeling branch, and time-series modeling and dimension normalization are executed sequentially to obtain a time-series branch feature vector sequence that corresponds one-to-one with the time window. The sequential execution of time-series modeling and dimension normalization refers to using a time-series convolutional network to extract the time-dependent features of the electrical signal, and unifying the output dimension through linear mapping and compression operations to form the corresponding time-series branch feature vector sequence. The aligned video frames or frame groups are input into the lightweight image branch, and convolution processing and downsampling are performed sequentially to obtain the image branch feature vector sequence corresponding to the time window. The sequential convolution processing and downsampling refer to extracting the local structural features of the video image in the spatial dimension through the lightweight convolutional network, and using downsampling operation after multiple convolutions to reduce the feature dimension and redundant information, forming the image branch feature vector sequence corresponding to the time window. Feature vector sequences of the time-frequency branch, time-series branch, and image branch are concatenated in the channel dimension according to the time window order. The concatenated feature vector sequences are then weighted and fused based on cross-modal interactive attention to generate a joint feature vector sequence. The weighted fusion of the concatenated feature vector sequences based on cross-modal interactive attention refers to establishing the interrelationship between different modal features, calculating the correlation weight of each modal feature at the time step, and then weighting and combining the concatenated feature vector sequences according to the weights.

[0025] In this embodiment, the step of inputting the joint feature vector sequence into the phase transition hierarchical decoder, outputting hierarchical recognition results according to the hierarchy, determining the phase transition stage time window, and outputting the opening curve includes: A phase-change hierarchical decoder is constructed, which consists of a first-layer state decoding layer, a second-layer stage decoding layer and a third-layer fault decoding layer. The joint feature vector sequence is input into the phase-change hierarchical decoder. In the first state decoding layer, the joint feature vector sequence is decoded in chronological order to output the open / closed state sequence, which includes open, closed and transition states. In the second-level decoding layer, based on a preset phase sequence mask, information transmission is only allowed between the start-up phase and the acceleration phase, the acceleration phase and the stabilization phase, the stabilization phase and the deceleration phase, and the deceleration phase and the limit contact phase. A monotonic gating rule is applied in conjunction with the start-up / stop state sequence and the actuator control signal to output the phase transition phase sequence and determine the phase transition phase time window. The preset phase sequence mask refers to: According to the physical sequence of the valve opening and closing process, all phase change stages are arranged in a fixed order of start-up, acceleration, stabilization, deceleration and limit contact, and a mask matrix is ​​constructed that only allows connections between adjacent stages. In the mask matrix, the connection weights of adjacent stages are preserved, while the connection weights of non-adjacent stages are reset to zero. This constrains the decoding process to pass information only between reasonable adjacent stages, avoiding unreasonable stage transitions such as jumps or reversals. In the third fault decoding layer, context routing is performed between the open / closed state sequence and the phase transition stage sequence in the stage-specific fault path, and the initial fault judgment label sequence corresponding to each phase transition stage is output. The initial fault judgment label sequence includes jamming, leakage, limit switch failure, actuator abnormality and normal. The opening regression head built into the phase change hierarchical decoder outputs an opening curve aligned with the phase change stage time window under the common constraints of the opening / closing state sequence and the phase change stage sequence. The opening / closing state sequence, the phase change stage sequence, and the initial fault label sequence are collectively referred to as the hierarchical identification result.

[0026] This invention constructs a phase-change hierarchical decoder, decomposing the valve opening and closing process into three levels: state, stage, and fault. It introduces a preset phase sequence mask and monotonic gating rules to ensure the reasonable order and physical consistency of phase-change stages during decoding, avoiding stage jumps or reverse transfers that are common in traditional methods. Through a context routing mechanism, specific fault identification paths can be selected at different phase-change stages, achieving a correspondence between fault types and stage states, thus improving the accuracy and interpretability of fault diagnosis. The built-in opening regression head outputs opening curves under the dual constraints of state and stage results, significantly reducing deviations from actual measurements. This not only enhances the precision of identification and diagnosis but also enables effective capture of early anomalies and unified output of multi-dimensional results, improving the reliability and practicality of valve condition monitoring.

[0027] In this embodiment, the opening regression head refers to a regression submodule set in the phase change layer decoder for outputting the valve opening change curve over time. The regression submodule takes the joint feature vector sequence, the opening and closing state sequence, the phase change stage sequence, and the actuator control signal as inputs, performs time alignment according to the phase change stage time window, applies monotonically non-decreasing gating in the opening stage, applies monotonically non-increasing gating in the closing stage, and performs boundary clamping in the limit contact stage, outputting an opening curve with a consistent sampling period and a value limited to the range of 0% to 100%.

[0028] In this embodiment, the output event modeling results include: The received phase transition stage sequence and phase transition stage time window, as well as the output joint feature vector sequence; For each phase transition stage time window, a feature subsequence aligned with the phase transition stage time window is extracted from the joint feature vector sequence to determine the event type as limit impact event, stuck micropulse event and leakage acoustic pattern cluster event. A phase transition condition point process layer is constructed based on feature subsequences. This layer includes a stage-specific triggering kernel library, event mutual inhibition and cooperative gating, cross-modal confidence weighting, and a multi-resolution temporal grid. The phase-specific triggering kernel libraries correspond to the startup phase, acceleration phase, stabilization phase, deceleration phase, and limit contact phase, respectively. Event mutual inhibition and collaborative gating constrain the triggering relationship of different event types within the same time window. They suppress unreasonable concurrent triggering through mutual exclusion mechanism and strengthen joint triggering with causal relationship through collaborative mechanism. Cross-modal confidence weighting weights event scores based on the feature confidence levels of each branch; Multi-resolution temporal raster simultaneously characterizes event triggering at both fine-grained and coarse-grained time scales; Within each phase transition stage time window, a neural Hawkes point process is used to model the arrival time and intensity of limiting impact events, stuck micropulse events, and leakage acoustic signature cluster events. This generates a candidate event timestamp set, outputs event intensity curves that change over time, and calculates the expected event count and the posterior distribution of event types. Specifically, the modeling of the arrival time and intensity of limiting impact events, stuck micropulse events, and leakage acoustic signature cluster events using a neural Hawkes point process involves: For limit impact events, the acoustic pulse characteristics and vibration impact characteristics when the valve approaches the limit position are used as trigger inputs. The neural Hawkes point process performs conditional modeling on the arrival intensity of the current time step based on the triggering of the previous event, and characterizes the concentrated burst characteristics and boundary alignment rules of limit impact events. For stuck micropulse events, abnormal small fluctuations in actuator control quantities and electrical signals are used as trigger inputs. The neural Hawkes point process captures the cumulative effect of multiple weak triggers within a short time scale. By modeling the influence of historical events on the intensity of the current time step, the high-frequency small-amplitude pulse pattern of stuck events is revealed. For leaked voiceprint cluster events, continuous frequency energy bands in acoustic signals and local morphological changes in video signals are used as trigger inputs. The neural Hawkes point process models the dependence of adjacent events over a longer time scale, generating trigger intensity curves that decay or increase over time, thus characterizing the clustered triggering features of leaked events. The calculation of the expected event count and the posterior distribution of event types is specifically as follows: Within each phase transition time window, the event intensity curve generated by the neural Hawkes point process is integrated in the time dimension to obtain the expected count of various events within the phase transition time window, which is used to characterize the statistical frequency of events. Based on the candidate event timestamp set, the intensity values ​​of different event types are normalized, the probability distribution of each event type at each time step is calculated, and the posterior distribution of the event type is obtained to distinguish different event categories within the same time window. The execution phase boundary alignment and limit calibration involves matching the candidate event timestamp set with the boundary of the phase transition phase time window, eliminating candidate event timestamps that fall outside the boundary, and determining the actual event time at the boundary through the alignment calibration of the limit impact event. Valid trigger points are then selected by combining the confidence scores of stuck micropulse events and leakage acoustic cluster events to obtain the event timestamp set. The event timestamp set, event intensity curve, expected event count, and posterior distribution of event type for each phase transition stage time window are summarized into the event modeling results.

[0029] This invention achieves high-precision capture and statistical modeling of transient abnormal events during the valve opening and closing phase transition by introducing a phase transition conditional point process layer and a neural Hawkes point process in the event modeling process. A phase-specific trigger kernel library ensures targeted extraction of event features corresponding to different phase transition stages. Event mutual inhibition and collaborative gating effectively constrain unreasonable concurrent triggering and highlight causal events. Cross-modal confidence weighting and multi-resolution temporal grids enhance the robustness and multi-scale adaptability of event representation. The neural Hawkes point process further conditionally models the arrival time and intensity of limit impact, stuck micro-pulse, and leakage acoustic cluster events in the time dimension. It can not only characterize their concentrated bursts, cumulative triggering, and clustering characteristics, but also output the expected event count and posterior type distribution, realizing a quantitative expression of event frequency and category probability. This design overcomes the limitations of existing technologies that can only rely on single signals or thresholds for detection, improving the sensitivity and diagnostic interpretability of early weak anomalies, thereby achieving refined monitoring and intelligent diagnosis of valve operating status.

[0030] In this embodiment, the output consistency verification result includes: Receive phase transition phase sequence, phase transition phase time window, opening curve, actuator control signal and multimodal timing sample, and perform time alignment according to the phase transition phase time window; A conditional reversible generation diagnostic tool is constructed, which consists of a conditional encoding unit, a reversible mapping unit, and a consistency evaluation unit, wherein: The reversible mapping unit is composed of multiple reversible coupled mapping layers connected in series. Each level contains a scaling transformation sub-layer and a translation transformation sub-layer while maintaining the consistency of input and output dimensions. The consistency evaluation unit consists of a log-likelihood evaluation module and a residual aggregation module. The log-likelihood evaluation module outputs the conditional log-likelihood sequence for each time step, and the residual aggregation module outputs the modal-level reconstructed residual spectrum. Within each phase transition stage time window, the conditional encoding unit combines the phase transition stage sequence, the stage identifier corresponding to the phase transition stage time window, the opening curve, and the actuator control signal into a conditional vector sequence according to the time step, and completes length normalization and channel alignment. For each time step, the multimodal time series samples and the condition vector sequence are input into the invertible mapping unit. Forward mapping is performed to obtain the internal representation, and inverse mapping is performed under the same condition vector to obtain the reconstructed segment, forming the reconstructed sequence corresponding to the time step. Specifically, the forward mapping to obtain the internal representation and the inverse mapping under the same condition vector are performed as follows: In the forward mapping process, the input multimodal time series samples are transformed layer by layer under the conditional vector constraint through coupled scaling and translation transformations, mapping high-dimensional observation data into low-dimensional invertible internal representations; In the reverse mapping process, the same condition vector is used as a constraint to restore the internal representation to the reconstructed fragment of the original data space layer by layer in the reverse transformation order, thereby ensuring the consistency between the input and the reconstruction. The consistency evaluation unit calculates the time-domain and frequency-domain residuals of the acoustic and vibration channels, the amplitude and morphological residuals of the electrical channel, and the spatial and temporal residuals of the video channel for each time step of the reconstructed sequence and the original sequence. These residuals are then summarized into a modal-level reconstruction residual spectrum. Simultaneously, the conditional log-likelihood sequence corresponding to each time step is output. The conditional log-likelihood sequence and the modal-level reconstruction residual spectrum are collectively referred to as the consistency verification result. Specifically, the calculation of the time-domain and frequency-domain residuals of the acoustic and vibration channels, the amplitude and morphological residuals of the electrical channel, and the spatial and temporal residuals of the video channel involves the following steps: The time-domain residuals of the acoustic and vibration channels are obtained by comparing the amplitude differences between the original waveform and the reconstructed waveform step by step, while the frequency-domain residuals are obtained by calculating the frequency energy distribution differences after performing time-frequency transformations on the original waveform and the reconstructed waveform respectively. The amplitude residual of the electrical channel is obtained by comparing the difference in current and voltage amplitudes between the original signal and the reconstructed signal at the same time step, and the morphological residual is obtained by extracting the change trend of the original signal and the reconstructed signal within the time window and comparing the differences. The spatial residual of the video channel is obtained by comparing the differences in the spatial distribution of pixels or feature layers between the original image and the reconstructed image frame by frame, while the temporal residual is obtained by analyzing the consistency of dynamic changes of video frame groups at adjacent time steps and calculating the differences.

[0031] This invention constructs a conditionally reversible generation diagnostic tool, introducing phase transition stage sequences, opening curves, and actuator control signals into conditional constraints. This enables reversible mapping and reconstruction of multimodal time-series samples, establishing bidirectional consistency between input and output during the diagnostic process. The reversible mapping unit allows for simultaneous acquisition of internal representations and reconstruction results under conditional vector constraints, maintaining the integrity of feature transformations and preventing information loss. The consistency evaluation unit, combined with a dual verification mechanism of conditional log-likelihood and modal-level residual spectrum, accurately quantifies the consistency and deviation between diagnostic results and original observations, achieving multi-level verification from global statistics to modal details. This design overcomes the limitations of traditional "black box" models, which lack interpretability and result verification, giving valve opening / closing states and fault diagnosis results higher reliability and interpretability, improving the sensitivity of early anomaly detection and the practicality of industrial applications.

[0032] In this embodiment, the fusion of the hierarchical identification results, event modeling results, and consistency verification results to obtain the valve opening / closing status, opening degree, phase transition stage, fault type, and severity score includes: The hierarchical identification results, event modeling results, and consistency verification results are time-aligned according to the phase transition stage time window under the same time base, and resampled according to a unified sampling period. The hierarchical identification confidence, event arrival rate and expected event count, log likelihood and modal residual are interval-standardized. Within each phase transition stage, based on a preset weight table and stage gating rules, the hierarchical identification results are mapped to state evidence, the event modeling results are mapped to event evidence, and the consistency verification results are mapped to consistency evidence. The three types of evidence are weighted and synthesized to generate stage-level fusion scores and modality-level fusion scores. The valve opening / closing state and phase transition stage are determined based on the stage-level fusion score and the modal-level fusion score. The fault type is then identified, the opening degree is obtained by combining the opening curve, and a severity score is calculated. Specifically, the calculation of the severity score involves: Within each phase transition stage time window, the stage-level fusion score and modal-level fusion score are used as the basic risk quantity. Combined with the event intensity, count expectation and type posterior in the event modeling results, preset weights and penalty coefficients are applied to leakage, jamming and limit impact respectively to obtain the normalized intra-stage risk value. The risk values ​​within the phase are corrected based on the consistency verification results: When the log-likelihood is low or the modal residual spectrum increases, adjust the magnitude upwards; when both are in the normal range, adjust the magnitude downwards. Then, the severity scores for each stage are obtained by weighting the duration and criticality of each stage. The severity scores of each stage are summarized according to stage weight and time sequence to form a baseline severity score for the entire process. The confidence level of the stratified identification results is then adjusted by confidence reduction or gain. Finally, the scores are mapped to the final severity score and corresponding level through a historical calibration table for alarm triggering and linkage control. The historical calibration table is a mapping reference table established based on a large amount of valve operation and maintenance data. It statistically calibrates the severity levels corresponding to different event intensities, residual levels, log-likelihood intervals and actual maintenance results, forming a correspondence between risk values ​​and severity scores.

[0033] Example 1

[0034] To verify the feasibility of this invention in practice, it was applied to the valve opening and closing status monitoring system of a coastal petrochemical refinery. The refinery has a daily crude oil processing capacity of 35,000 tons, with key processes including hydrocracking, atmospheric and vacuum distillation, reforming, and coking. The entire plant has a total of 1432 electric, pneumatic, and hydraulic valves, of which approximately 156 are main valves on critical production lines. These valves operate frequently and are responsible for regulating feedstock flow and controlling temperature and pressure. Any untimely valve opening or closing, jamming, or leakage will directly impact production safety and economic efficiency.

[0035] Previously, the plant primarily used a single current sensor or motor current curve to determine the valve's on / off status. In actual operation, threshold drift and misjudgments frequently occurred. For example, under high-temperature conditions, large fluctuations in motor load could easily cause valve jamming and misjudgments; under low-temperature and high-humidity conditions, acoustic signal attenuation was significant, leading to insensitive leak detection. According to the plant's maintenance records, in 2024, there were 22 production interruptions due to misjudgments of valve status, with an average downtime of 3.5 hours per incident, resulting in direct economic losses of approximately 4.7 million yuan.

[0036] After introducing the deep learning-based valve opening and closing status identification and diagnosis method proposed in this invention, it was first deployed on the key rebalancing device. The monitoring targets were 20 main electric control valves, each equipped with acoustic, vibration, electrical, and video acquisition devices, which were connected to the monitoring system proposed in this invention along with the actuator control signals. The system achieves millisecond-level synchronization using a unified time base and performs denoising, segmentation, and normalization processing on the acquired signals. Acoustic and vibration signals are transformed by time-frequency conversion to generate time-spectrum diagrams, while electrical and video signals are normalized in amplitude and shape to form multimodal time-series samples.

[0037] Subsequently, the system performs branch modeling on the time-spectrum graph, electrical segments, and video frames through a cross-modal feature extraction network, fuses them to generate a joint feature vector sequence, and inputs it into a phase-change hierarchical decoder. This decoder sequentially outputs the opening / closing state sequence, the phase-change stage sequence, and the initial fault judgment result, and generates an opening curve aligned with the phase-change stage window using a built-in opening regression head. In a single valve operation from fully closed to fully open, the system accurately distinguishes five stages: startup, acceleration, stabilization, deceleration, and limit contact. The average deviation between the opening curve and the manual measurement results on-site does not exceed 1.8%.

[0038] In terms of anomaly detection, this invention utilizes neural Hawkes point processes to model limit impacts, stuck micropulses, and leakage acoustic clusters. During a reformer switchover in March 2025, the system accurately captured a stuck micropulse event at the 3.4-second mark of the valve and detected a slight leakage acoustic cluster during the limit contact phase. On-site inspection revealed a loose valve packing gland with a leakage rate of approximately 0.3 L / min. The system alarm and inspection results were highly consistent, demonstrating the high sensitivity of this invention to early-stage, latent faults.

[0039] To verify the effectiveness of the present invention, the manufacturer conducted a comparative test from March to June 2025, and statistically analyzed the status identification and fault detection performance of 20 valves during daily opening and closing.

[0040] Table 1 Comparison of Valve Status Identification and Fault Detection Data Index category Conventional method value Invention method value Promotion effect State recognition accuracy 87.2% 96.8% +9.6% Stuck event detection rate 59.7% 91.6% +31.9% Leakage event detection rate 43.5% 88.3% +44.8% False alarm rate 12.4% 3.1% -9.3% Average alarm confirmation rate 62.8% 88.2% +25.4% Opening curve deviation mean value 6.4% 1.8% -4.6% As can be seen from the data in Table 1, the present invention significantly improves valve opening and closing status identification and fault diagnosis compared to traditional methods. In terms of overall status identification accuracy, the traditional method only achieves 87.2%, while the method of the present invention reaches 96.8%, an improvement of 9.6 percentage points. This demonstrates that the present invention, through multimodal feature fusion and hierarchical decoding mechanisms, can more comprehensively and accurately capture the characteristic information of the valve operation process.

[0041] In terms of anomaly detection, the advantages of this invention are even more pronounced. For stuck events, the detection rate of traditional methods is less than 60%, only 59.7%, while the detection rate of this invention is as high as 91.6%, an improvement of 31.9 percentage points, demonstrating the high sensitivity of the neural Hawkes point process to transient anomalies. For leakage events, the detection rate of traditional methods is only 43.5%, while this invention improves it to 88.3%, an improvement of 44.8 percentage points. This indicates that through the joint analysis of acoustic, vibration, and video signals, this invention can effectively identify hidden leaks that are difficult to detect by traditional methods.

[0042] In terms of system reliability, this invention also shows significant improvements. Traditional methods have a false alarm rate as high as 12.4%, easily leading to unnecessary maintenance and downtime, while the false alarm rate of this invention is only 3.1%, a reduction of 9.3 percentage points, indicating that the consistency verification mechanism greatly improves the reliability of diagnostic results. Simultaneously, the average alarm confirmation rate increased from 62.8% to 88.2%, an improvement of 25.4 percentage points, indicating that the alarms issued by the system are more reliable and can be effectively verified by on-site maintenance results. Finally, regarding the accuracy of the opening curve, traditional methods have an average deviation of 6.4% from actual measurement results, while this invention reduces the deviation to 1.8%, indicating that the synergistic effect of the opening regression head and the hierarchical decoder significantly improves the quantification accuracy during valve opening and closing.

[0043] This invention not only surpasses traditional methods in recognition accuracy, but also achieves breakthrough improvements in key indicators such as abnormal event detection, alarm reliability, and opening curve accuracy, fully demonstrating the feasibility and practical value of this invention under complex working conditions.

[0044] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for valve opening and closing status recognition and diagnosis based on deep learning, characterized in that, include: Acoustic signals, vibration signals, electrical signals, video signals, and actuator control signals are collected during valve operation. Based on a unified time reference, millisecond-level synchronization is performed, followed by denoising, segmentation, and normalization. Time-frequency transformation is performed on acoustic and vibration signals to obtain time-spectrum diagrams, and amplitude and morphology are standardized for electrical and video signals to form multimodal time-series samples. The time-frequency spectrum is input into the time-frequency convolution branch, the electrical signal is input into the time-series modeling branch, and the video signal is input into the lightweight image branch. The branch features are extracted respectively, and a joint feature vector sequence is generated by cross-modal interactive attention fusion. The joint feature vector sequence is input into the phase transition hierarchical decoder, and the hierarchical recognition results are output according to the hierarchy. The time window of the phase transition stage is determined and the opening curve is output. Within the phase transition phase time window, a phase transition condition point process layer is constructed based on the joint feature vector sequence. The arrival time and occurrence intensity of limit impact events, stuck micropulse events and leakage acoustic pattern cluster events are modeled using neural Hawkes point processes, and the event modeling results are output. Using the phase transition stage sequence, opening curve, and actuator control signal as a condition set, a condition reversible generation diagnostic tool is constructed to perform condition generation and reversible reconstruction on multimodal time series samples and output consistency verification results. The results of hierarchical identification, event modeling, and consistency verification are integrated to obtain the valve opening and closing status, opening degree, phase change stage, fault type, and severity score. When the preset threshold is exceeded, an alarm is triggered and a linkage control command is issued. At the same time, drift monitoring and recalibration are performed and the diagnostic results are archived.

2. The valve opening and closing state recognition and diagnosis method based on deep learning according to claim 1, characterized in that, The process of performing millisecond-level synchronization based on a unified time reference, and performing denoising, segmentation and normalization, refers to aligning acoustic signals, vibration signals, electrical signals, video signals and actuator control signals according to the same time reference, using filtering to remove high-frequency noise and DC components, dividing continuous signals into multiple time segments according to a preset sampling length, and unifying different modal signals to a comparable scale through amplitude standardization and mean-variance normalization.

3. The valve opening and closing state recognition and diagnosis method based on deep learning according to claim 1, characterized in that, The process of performing time-frequency transformation on acoustic and vibration signals to obtain time-frequency spectrograms, and the process of amplitude and shape standardization on electrical and video signals, refers to obtaining frequency distribution characteristics and forming two-dimensional time-frequency spectrograms by transforming acoustic and vibration signals from the time domain to the frequency domain within a fixed time window, eliminating amplitude differences between different measurement channels by normalizing the amplitude of electrical signals, and performing shape standardization on video signals by scaling and equalizing grayscale distribution.

4. The valve opening and closing state recognition and diagnosis method based on deep learning according to claim 1, characterized in that, The generation of a joint feature vector sequence through cross-modal interactive attention fusion includes: From the formed multimodal time-series samples, the time spectrum of acoustics and vibration, the time segments of electrical signals, and video frames or frame groups aligned with the time segments are obtained according to a unified time window. The time-frequency spectrum is input into the time-frequency convolution branch in channel order, and convolution processing and feature compression are performed sequentially to obtain a time-frequency branch feature vector sequence that corresponds one-to-one with the time window. The time segment of the electrical signal is input into the timing modeling branch, and timing modeling and dimension regularization are performed sequentially to obtain a sequence of timing branch feature vectors that correspond one-to-one with the time window. The aligned video frames or frame groups are input into the lightweight image branch, and convolution and downsampling are performed sequentially to obtain a sequence of image branch feature vectors that correspond one-to-one with the time window. The feature vector sequences of the time-frequency branch, the time-series branch, and the image branch are concatenated in the channel dimension according to the time window order. The concatenated feature vector sequences are then weighted and fused based on cross-modal interactive attention to generate a joint feature vector sequence.

5. The valve opening and closing state recognition and diagnosis method based on deep learning according to claim 1, characterized in that, The step of inputting the joint feature vector sequence into the phase transition hierarchical decoder, outputting hierarchical recognition results according to the hierarchy, determining the time window of the phase transition stage, and outputting the opening curve includes: A phase-change hierarchical decoder is constructed, which consists of a first-layer state decoding layer, a second-layer stage decoding layer and a third-layer fault decoding layer. The joint feature vector sequence is input into the phase-change hierarchical decoder. In the first state decoding layer, the joint feature vector sequence is decoded in chronological order to output the open and closed state sequence; In the second-level stage decoding layer, based on the preset phase sequence mask, only the information transmission between the start-up stage and the acceleration stage, the acceleration stage and the stabilization stage, the stabilization stage and the deceleration stage, and the deceleration stage and the limit contact stage is allowed. In addition, a monotonic gating rule is applied in combination with the start-up and stop state sequence and the actuator control signal to output the phase transition stage sequence and determine the phase transition stage time window. In the third fault decoding layer, context routing is performed between the open / closed state sequence and the phase transition stage sequence in the stage-specific fault path, and the initial fault judgment label sequence corresponding to each phase transition stage is output. The opening regression head built into the phase change hierarchical decoder outputs an opening curve aligned with the phase change stage time window under the common constraints of the opening / closing state sequence and the phase change stage sequence. The opening / closing state sequence, the phase change stage sequence, and the initial fault label sequence are collectively referred to as the hierarchical identification result.

6. The valve opening and closing state recognition and diagnosis method based on deep learning according to claim 5, characterized in that, The opening regression head refers to the regression submodule set in the phase change layer decoder to output the valve opening change curve over time. The regression submodule takes the joint feature vector sequence, the opening and closing state sequence, the phase change stage sequence, and the actuator control signal as input. It performs time alignment according to the phase change stage time window, applies monotonically non-decreasing gating in the opening stage, applies monotonically non-increasing gating in the closing stage, and performs boundary clamping in the limit contact stage. It outputs an opening curve with a consistent sampling period and a value limited to the range of 0% to 100%.

7. The valve opening and closing state recognition and diagnosis method based on deep learning according to claim 1, characterized in that, The output event modeling results include: The received phase transition stage sequence and phase transition stage time window, as well as the output joint feature vector sequence; For each phase transition stage time window, a feature subsequence aligned with the phase transition stage time window is extracted from the joint feature vector sequence to determine the event type as limit impact event, stuck micropulse event and leakage acoustic pattern cluster event. A phase transition condition point process layer is constructed based on feature subsequences. This layer includes a stage-specific triggering kernel library, event mutual inhibition and cooperative gating, cross-modal confidence weighting, and a multi-resolution temporal grid. The phase-specific triggering kernel libraries correspond to the startup phase, acceleration phase, stabilization phase, deceleration phase, and limit contact phase, respectively. Event mutual inhibition and collaborative gating constrain the triggering relationship of different event types within the same time window. They suppress unreasonable concurrent triggering through mutual exclusion mechanism and strengthen joint triggering with causal relationship through collaborative mechanism. Cross-modal confidence weighting weights event scores based on the feature confidence levels of each branch; Multi-resolution temporal raster simultaneously characterizes event triggering at both fine-grained and coarse-grained time scales; Within each phase transition stage time window, the arrival time and intensity of limit impact events, stuck micropulse events and leakage acoustic cluster events are modeled using neural Hawkes point processes, generating a candidate event timestamp set, outputting the event intensity curve that changes with time, and calculating the expected event count and the posterior distribution of event types. The execution phase boundary alignment and limit calibration involves constraining and matching the candidate event timestamp set with the boundary of the phase transition phase time window, eliminating candidate event timestamps that fall outside the boundary, and determining the actual event time at the boundary through the alignment calibration of the limit impact event. Valid trigger points are then selected by combining the confidence scores of stuck micropulse events and leakage acoustic cluster events to obtain the event timestamp set. The event timestamp set, event intensity curve, expected event count, and posterior distribution of event type for each phase transition stage time window are summarized into the event modeling results.

8. The valve opening and closing state recognition and diagnosis method based on deep learning according to claim 1, characterized in that, The output consistency verification results include: Receive phase transition phase sequence, phase transition phase time window, opening curve, actuator control signal and multimodal timing sample, and perform time alignment according to the phase transition phase time window; A conditional reversible generation diagnostic tool is constructed, which consists of a conditional encoding unit, a reversible mapping unit, and a consistency evaluation unit, wherein: The reversible mapping unit is composed of multiple reversible coupled mapping layers connected in series. Each level contains a scaling transformation sub-layer and a translation transformation sub-layer while maintaining the consistency of input and output dimensions. The consistency evaluation unit consists of a log-likelihood evaluation module and a residual aggregation module. The log-likelihood evaluation module outputs the conditional log-likelihood sequence for each time step, and the residual aggregation module outputs the modal-level reconstructed residual spectrum. Within each phase transition stage time window, the conditional encoding unit combines the phase transition stage sequence, the stage identifier corresponding to the phase transition stage time window, the opening curve, and the actuator control signal into a conditional vector sequence according to the time step, and completes length normalization and channel alignment. For each time step, the multimodal time series samples and the condition vector sequence are input into the invertible mapping unit, and forward mapping is performed to obtain the internal representation. Then, inverse mapping is performed under the same condition vector to obtain the reconstructed fragment, forming the reconstructed sequence corresponding to the time step. The consistency evaluation unit calculates the time-domain and frequency-domain residuals of the acoustic and vibration channels, the amplitude and morphological residuals of the electrical channels, and the spatial and temporal residuals of the video channels for each time step of the reconstructed sequence and the original sequence. These residuals are then summarized into a modal-level reconstruction residual spectrum. Simultaneously, the conditional log-likelihood sequence corresponding to each time step is output. The conditional log-likelihood sequence and the modal-level reconstruction residual spectrum are collectively referred to as the consistency verification result.

9. The valve opening and closing state recognition and diagnosis method based on deep learning according to claim 1, characterized in that, The fusion of hierarchical identification results, event modeling results, and consistency verification results yields valve opening / closing status, opening degree, phase transition stage, fault type, and severity score, including: The hierarchical identification results, event modeling results, and consistency verification results are time-aligned according to the phase transition stage time window under the same time base, and resampled according to a unified sampling period. The hierarchical identification confidence, event arrival rate and expected event count, log likelihood and modal residual are interval-standardized. Within each phase transition stage, based on a preset weight table and stage gating rules, the hierarchical identification results are mapped to state evidence, the event modeling results are mapped to event evidence, and the consistency verification results are mapped to consistency evidence. The three types of evidence are weighted and synthesized to generate stage-level fusion scores and modality-level fusion scores. The valve opening and closing status and phase transition stage are determined based on the stage-level fusion score and the modal-level fusion score, the fault type is determined, the opening degree is obtained by combining the opening curve, and the severity score is calculated.

Images