A valve opening and closing state recognition and diagnosis method based on deep learning

Abstract

The application discloses a valve opening and closing state recognition and diagnosis method based on deep learning, comprising the following steps: collecting and synchronizing multiple source signals to generate standardized multi-modal time sequence samples; extracting modal features by multiple branches and fusing them into a joint feature vector sequence; inputting the joint features into a phase change layered decoder to output layered recognition results; constructing a point process layer to model multiple abnormal events in a phase change stage and output event modeling results; constructing a conditional reversible generation diagnostic device to output consistency checking results; and fusing the results to output state and diagnosis information and execute alarm and archiving. Through multi-modal deep learning feature fusion, phase change layered decoding and conditional modeling, the application realizes accurate recognition of the valve opening and closing state, fine division of the phase change stage and intelligent diagnosis of early faults.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of industrial equipment state monitoring, and in particular to a valve opening and closing state recognition and diagnosis method based on deep learning. BACKGROUND

[0002] In industrial automation and process control systems, valves, as key execution components, are widely used in petrochemical, power, metallurgy, water conservancy and other fields. The accurate recognition and fault diagnosis of the opening and closing state of the valve is directly related to the safety and stability of the production process. In the prior art, the recognition method of the opening and closing state of the valve depends on a single sensor signal, such as current, voltage or pressure signal, and the state is distinguished by setting a threshold or pattern matching. Although this method is simple to implement, it can only coarsely distinguish between open and closed states of the valve, and cannot finely capture the dynamic characteristics of the phase change stage in the opening and closing process, especially in complex working conditions, which is prone to misjudgment and omission. The single signal is sensitive to environmental noise and equipment aging, and the diagnosis result lacks robustness and credibility.

[0003] With the development of deep learning, some studies have begun to try to use convolutional neural networks, recurrent neural networks and other models to analyze sensor signals, so as to realize intelligent recognition of the valve state. However, this method usually only processes single-mode data, and fails to fully utilize the complementary information between multi-modal data such as acoustics, vibration, electricity and video, resulting in obvious deficiencies in capturing transient abnormal events (such as stuck micro-pulse, leakage sound cluster) in the phase change stage of the valve opening and closing. The existing technology lacks a consistency checking mechanism between the recognition result and the input observation in the generation of the diagnosis process, making it difficult to trace and explain the diagnosis result, and difficult to meet the requirements of high confidence and explainability in industrial applications.

[0004] Therefore, the existing valve opening and closing state recognition technology has the following defects: first, the recognition granularity is insufficient, only the open and closed states can be judged, and there is a lack of fine recognition of the phase change stage; second, there is a lack of modeling ability for transient abnormal events, making it difficult to find early fault hidden dangers in time; third, there is a lack of multi-modal consistency checking mechanism, resulting in insufficient diagnosis accuracy and explainability.

[0005] Therefore, how to provide a valve opening and closing state recognition and diagnosis method based on deep learning is a problem that those skilled in the art need to solve. SUMMARY

[0006] One purpose of the present application is to propose a valve opening and closing state recognition and diagnosis method based on deep learning, which fully utilizes multi-modal deep feature extraction, phase change hierarchical decoding and neural point process modeling technology, and constructs a conditional reversible generation diagnostic device to realize comprehensive recognition and checking of valve opening and closing state, phase change stage, transient abnormal event and fault type. The present application describes in detail the overall process of multi-modal data acquisition and synchronization, feature extraction and cross-modal fusion, hierarchical decoder output opening and closing state and phase change stage, neural Hawkes point process modeling of transient events, and consistency checking of conditional reversible generation diagnostic device, which can realize fine identification of valve opening and closing state and accurate diagnosis of early faults under complex working conditions. The method has the advantages of high recognition accuracy, sensitivity to transient abnormal events, interpretable diagnosis results, strong robustness and adaptability to complex working conditions.

[0007] According to the valve opening and closing state recognition and diagnosis method based on deep learning, the method comprises the following steps:

[0008] Acquire acoustic signals, vibration signals, electrical signals, video signals and actuator control signals during valve operation, synchronize at the millisecond level based on a unified time reference, perform denoising, segmentation and normalization processing, perform time-frequency transformation on the acoustic signals and vibration signals to obtain time-frequency spectrograms, and perform amplitude and morphology standardization on the electrical signals and video signals to form multi-modal time series samples;

[0009] Input the time-frequency spectrogram into the time-frequency convolution branch, input the electrical signal into the time series modeling branch, and input the video signal into the lightweight image branch to respectively extract branch features, and generate a joint feature vector sequence through cross-modal interaction attention fusion;

[0010] Input the joint feature vector sequence into the phase change hierarchical decoder, output hierarchical recognition results by layer, determine the phase change stage time window and output the opening degree curve;

[0011] In the phase change stage time window, construct a phase change conditional point process layer based on the joint feature vector sequence, model the arrival time and occurrence intensity of the limit impact event, the stuck micro-pulse event and the leakage voiceprint cluster event using the neural Hawkes point process, and output the event modeling result;

[0012] Construct a conditional reversible generation diagnostic device using the phase change stage sequence and the opening degree curve, and the actuator control signal to form a condition set, perform conditional generation and reversible reconstruction on the multi-modal time series samples, and output the consistency checking result;

[0013] The layered identification result, event modeling result and consistency checking result are fused to obtain a valve opening and closing state, an opening degree, a phase change stage, a fault type and a severity score, and when exceeding a preset threshold, an alarm is triggered and a linkage control instruction is issued, and drift monitoring and recalibration are performed and diagnostic results are archived.

[0014] Optionally, the millisecond-level synchronization based on the unified time reference, the denoising, the segmentation and the normalization processing refer to time alignment of acoustic signals, vibration signals, electrical signals, video signals and actuator control quantity signals according to the same time reference, removal of high-frequency noise and direct current components by filtering processing, division of continuous signals into multiple time segments according to a preset sampling length, and unification of different modal signals to a comparable scale through amplitude standardization and mean-variance normalization.

[0015] Optionally, the time-frequency transformation of the acoustic signals and the vibration signals to obtain a time-frequency spectrum, and the amplitude and morphology standardization of the electrical signals and the video signals refer to obtaining frequency distribution characteristics and forming a two-dimensional time-frequency spectrum through a time-domain to frequency-domain transformation method within a fixed time window for the acoustic signals and the vibration signals, amplitude normalization of the electrical signals to eliminate amplitude differences of different measurement channels, and morphology standardization processing of the video signals through size scaling and gray scale distribution equalization.

[0016] Optionally, the generation of the joint feature vector sequence through cross-modal interactive attention fusion includes:

[0017] From the formed multi-modal time sequence samples, time-frequency spectrum of acoustic and vibration, time segment of electrical signal and video frame or frame group aligned with the time segment are obtained according to a unified time window;

[0018] The time-frequency spectrum is input into a time-frequency convolution branch in channel order, and convolution processing and feature compression are sequentially performed to obtain a time-frequency branch feature vector sequence corresponding to the time window one by one;

[0019] The time segment of the electrical signal is input into a time sequence modeling branch, and time sequence modeling and dimension regularization are sequentially performed to obtain a time sequence branch feature vector sequence corresponding to the time window one by one;

[0020] The aligned video frame or frame group is input into a lightweight image branch, and convolution processing and down-sampling are sequentially performed to obtain an image branch feature vector sequence corresponding to the time window one by one;

[0021] The feature vector sequences of the time-frequency branch, the time sequence branch and the image branch are spliced in channel dimension according to time window order, the spliced feature vector sequence is weighted and fused based on cross-modal interactive attention, and a joint feature vector sequence is generated.

[0022] Optionally, the step of inputting the joint feature vector sequence into the phase change layered decoder, outputting a layered recognition result, determining a phase change stage time window, and outputting an opening degree curve comprises:

[0023] The phase change layered decoder is constructed, and the phase change layered decoder is composed of a first layer state decoding layer, a second layer stage decoding layer, and a third layer fault decoding layer.

[0024] In the first layer state decoding layer, the joint feature vector sequence is decoded in time sequence, and an on-off state sequence is outputted.

[0025] In the second layer stage decoding layer, based on a preset phase sequence mask, only information transmission between the start-up stage and the acceleration stage, the acceleration stage and the stable stage, the stable stage and the deceleration stage, and the deceleration stage and the limit contact stage is allowed, and a monotonic gating rule is applied in combination with the on-off state sequence and the actuator control quantity signal, a phase change stage sequence is outputted, and a phase change stage time window is determined.

[0026] In the third layer fault decoding layer, context routing is performed between the on-off state sequence and the phase change stage sequence in the stage-specific fault path, and a fault preliminary judgment label sequence corresponding to each phase change stage is outputted.

[0027] An opening degree regression head built in the phase change layered decoder outputs an opening degree curve aligned with the phase change stage time window under the joint constraint of the on-off state sequence and the phase change stage sequence, and the on-off state sequence, the phase change stage sequence, and the fault preliminary judgment label sequence are collectively referred to as a layered recognition result.

[0028] Optionally, the opening degree regression head refers to a regression submodule arranged in the phase change layered decoder for outputting a valve opening degree curve varying with time, and the regression submodule takes the joint feature vector sequence, the on-off state sequence, the phase change stage sequence, and the actuator control quantity signal as input, performs time alignment according to the phase change stage time window, applies a monotonic non-decreasing gating in the opening stage, applies a monotonic non-increasing gating in the closing stage, and performs boundary clamping in the limit contact stage, and outputs an opening degree curve with consistent sampling periods and values limited in the range of 0% to 100%.

[0029] Optionally, the step of outputting the event modeling result comprises:

[0030] The received phase change stage sequence and phase change stage time window, and the outputted joint feature vector sequence are received.

[0031] For each phase change stage time window, a feature subsequence aligned with the phase change stage time window is intercepted from the joint feature vector sequence, and the event type is determined as a limit impact event, a stuck micro-pulse event, and a leakage voiceprint cluster event.

[0032] constructing a phase change condition point process layer based on the feature subsequence, the phase change condition point process layer comprising a stage-specific trigger core library, event mutual inhibition and synergistic gating, cross-modal confidence weighting and multi-resolution time grid, wherein:

[0033] The stage-specific trigger core library corresponds to the start stage, the acceleration stage, the stable stage, the deceleration stage and the limit contact stage respectively.

[0034] The event mutual inhibition and synergistic gating constrains the trigger relationship of different event types within the same time window, suppresses unreasonable concurrent triggers through mutual exclusion mechanism, and strengthens the joint trigger of events with causal correlation through synergistic mechanism.

[0035] The cross-modal confidence weighting weights the event score according to the feature confidence of each branch.

[0036] The multi-resolution time grid represents event triggering on both fine-grained and coarse-grained time scales.

[0037] In each phase change stage time window, a neural Hawkes point process is used to model the arrival time and intensity of limit impact events, stuck micro-pulse events and leakage voiceprint cluster events, generate a candidate event timestamp set, output an event intensity curve changing with time, and calculate event count expectation and event type posterior distribution.

[0038] The phase boundary alignment and limit calibration constrain and match the candidate event timestamp set with the boundary of the phase change stage time window, eliminate the candidate event timestamp falling outside the boundary, and determine the real event time at the boundary through the alignment and calibration of the limit impact event, combine the confidence score of the stuck micro-pulse event and the leakage voiceprint cluster event to filter the effective trigger points, and obtain the event timestamp set.

[0039] The event timestamp set, event intensity curve, event count expectation and event type posterior distribution of each phase change stage time window are summarized as event modeling results.

[0040] Optionally, the output consistency checking result comprises:

[0041] Receiving the phase change stage sequence, the phase change stage time window, the opening curve, the actuator control signal and the multi-modal time sequence sample, and performing time alignment according to the phase change stage time window.

[0042] Constructing a conditionally reversible generation diagnostic device, the conditionally reversible generation diagnostic device comprising a condition encoding unit, a reversible mapping unit and a consistency evaluation unit, wherein:

[0043] The reversible mapping unit is composed of multiple reversible coupling mapping layers in series, each of which contains a scale transformation sublayer and a translation transformation sublayer and keeps the input and output dimensions consistent;

[0044] The consistency evaluation unit is composed of a log-likelihood evaluation module and a residual aggregation module. The log-likelihood evaluation module outputs a conditional log-likelihood sequence at each time step, and the residual aggregation module outputs a modal-level reconstruction residual spectrum.

[0045] In each phase transition stage time window, the phase transition stage sequence, the phase identifier corresponding to the phase transition stage time window, the opening curve and the actuator control quantity signal are combined into a conditional vector sequence by the conditional encoding unit according to the time step, and the length is normalized and the channel is aligned.

[0046] For each time step, the multi-modal time sequence sample and the conditional vector sequence are input into the reversible mapping unit to perform forward mapping to obtain an internal representation, and inverse mapping is performed under the same conditional vector to obtain a reconstruction segment, forming a reconstruction sequence corresponding to the time step.

[0047] The reconstruction sequence and the original sequence are calculated by the consistency evaluation unit at each time step, and the time domain residual and the frequency domain residual of the acoustic and vibration channels, the amplitude residual and the morphology residual of the electrical channel, and the spatial residual and the time sequence residual of the video channel are calculated, and are summarized into a modal-level reconstruction residual spectrum. The conditional log-likelihood sequence corresponding to the time step is output, and the conditional log-likelihood sequence and the modal-level reconstruction residual spectrum are collectively referred to as consistency checking results.

[0048] Optionally, the layered identification results, event modeling results and consistency checking results are fused to obtain the valve opening and closing state, opening degree, phase transition stage, fault type and severity score, including:

[0049] The layered identification results, event modeling results and consistency checking results are time-aligned according to the phase transition stage time window and resampled according to a unified sampling period under the same time reference, and the layered identification confidence, event arrival rate and event count expectation, log-likelihood and modal-level residual are interval standardized.

[0050] In each phase transition stage, the layered identification results are mapped to state evidence, the event modeling results are mapped to event evidence, and the consistency checking results are mapped to consistency evidence according to the preset weight table and stage gating rules. The three types of evidence are weighted and synthesized to generate stage-level fusion scores and modal-level fusion scores.

[0051] The valve opening and closing state and the phase transition stage are determined according to the stage-level fusion scores and the modal-level fusion scores, the fault type is determined, the opening degree is obtained in combination with the opening curve, and the severity score is calculated.

[0052] The beneficial effects of the present application are:

[0053] The application realizes the synchronous analysis of multi-source data such as acoustics, vibration, electricity and video by introducing a multi-modal deep learning feature extraction and cross-modal interaction fusion mechanism in the valve opening and closing process. Compared with the existing method which relies on single signal threshold determination, the application can capture more rich opening and closing dynamic characteristics, and form a refined state division in the phase change stage, thereby effectively solving the limitation that the traditional technology can only coarsely distinguish open and closed, and greatly improving the accuracy and stability of valve opening and closing state recognition.

[0054] The application can not only output the opening and closing state, phase change stage and fault preliminary judgment result layer by layer by constructing a recognition framework combining a phase change layered decoder and a neural Hawkes point process, but also can model transient abnormal events such as limit impact, stagnation micro-pulse and leakage soundprint cluster in the phase change stage time window. This innovative design enables the system to identify potential hidden faults in the early stage, improves the sensitivity and detection rate of abnormal working conditions, and enhances the forward-looking and timeliness of valve operation state monitoring.

[0055] The conditional reversible generation diagnostic device proposed by the application outputs the logarithmic likelihood sequence and the modal level residual spectrum by reconstructing and checking the multi-modal time sequence samples under the given conditions, effectively establishes the consistency constraint between the diagnostic results and the original observations, improves the reliability and explainability of the diagnostic results, so that the valve state recognition and fault diagnosis are no longer just "black box" output, but have traceable and reliable basis, providing higher safety and robustness for industrial automation operation under complex working conditions. BRIEF DESCRIPTION OF DRAWINGS

[0056] The accompanying drawings are included to provide a further understanding of the application, and constitute a part of the specification, together with the embodiments of the application, to explain the application, and do not constitute a limitation on the application. In the drawings:

[0057] Figure 1 A flowchart of a valve opening and closing state recognition and diagnosis method based on deep learning proposed by the application;

[0058] Figure 2 A three-layer decoding structure diagram of a phase change layered decoder of a valve opening and closing state recognition and diagnosis method based on deep learning proposed by the application. DETAILED DESCRIPTION

[0059] The application will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, and only schematically show the basic structure of the application, and therefore only show the components related to the application.

[0060] REFERENCE Figure 1 and Figure 2A valve opening and closing state recognition and diagnosis method based on deep learning, comprising:

[0061] Acoustic signals, vibration signals, electrical signals, video signals and actuator control signals during valve operation are collected, millisecond-level synchronization is performed based on a unified time reference, denoising, segmentation and normalization processing are performed, time-frequency transformation is performed on the acoustic signals and vibration signals to obtain time-frequency spectrograms, and amplitude and morphology standardization is performed on the electrical signals and video signals to form multi-modal time series samples.

[0062] The time-frequency spectrograms are input into a time-frequency convolution branch, the electrical signals are input into a time series modeling branch, and the video signals are input into a lightweight image branch to extract branch features respectively, and a joint feature vector sequence is generated through cross-modal interaction attention fusion.

[0063] The joint feature vector sequence is input into a phase change hierarchical decoder, a hierarchical recognition result is output according to layers, a phase change stage time window is determined, and an opening degree curve is output.

[0064] Within the phase change stage time window, a phase change conditional point process layer is constructed based on the joint feature vector sequence, a neural Hawkes point process is used to model the arrival time and occurrence intensity of limit impact events, stuck micro-pulse events and leakage voiceprint cluster events, and an event modeling result is output.

[0065] The phase change stage sequence and the opening degree curve, and the actuator control signal constitute a condition set, a condition reversible generation diagnostic device is constructed, the multi-modal time series samples are conditionally generated and reversibly reconstructed, and a consistency checking result is output.

[0066] The hierarchical recognition result, the event modeling result and the consistency checking result are fused to obtain the valve opening and closing state, the opening degree, the phase change stage, the fault type and the severity score, and when the preset threshold is exceeded, an alarm is triggered and a linkage control instruction is issued, and drift monitoring and recalibration are performed and the diagnosis result is archived.

[0067] In the embodiment, the millisecond-level synchronization based on the unified time reference, the denoising, the segmentation and the normalization processing refer to time alignment of the acoustic signals, the vibration signals, the electrical signals, the video signals and the actuator control signals according to the same time reference, removal of high-frequency noise and direct current components by filtering processing, division of continuous signals into multiple time segments according to a preset sampling length, and unification of different modal signals to a comparable scale through amplitude standardization and mean-variance normalization.

[0068] In the embodiment, the time-frequency spectrum of the acoustic signal and the vibration signal is obtained by time-frequency transformation, and the amplitude and the shape of the electrical signal and the video signal are normalized, which means that the acoustic signal and the vibration signal are transformed from time domain to frequency domain in a fixed time window to obtain frequency distribution characteristics and form a two-dimensional time-frequency spectrum, the electrical signal is normalized by amplitude to eliminate the amplitude difference of different measurement channels, and the video signal is standardized by size scaling and gray distribution equalization.

[0069] In the embodiment, the joint feature vector sequence is generated by cross-modal interaction attention fusion, including:

[0070] From the formed multi-modal time sequence sample, the time-frequency spectrum of the acoustic and vibration, the time segment of the electrical signal, and the video frame or frame group aligned with the time segment are obtained in a unified time window;

[0071] The time-frequency spectrum is input into the time-frequency convolution branch in channel order, and convolution processing and feature compression are sequentially performed to obtain a time-frequency branch feature vector sequence corresponding to the time window one by one, wherein the sequential convolution processing and feature compression means that a two-dimensional convolution kernel is used to perform local weighted calculation along the time dimension and the frequency dimension, extract local energy distribution characteristics, and output enhanced time-frequency patterns through a nonlinear activation function, and pooling and dimension reduction operations are performed on feature mapping to remove redundant information and retain main discriminative features;

[0072] The time segment of the electrical signal is input into the time series modeling branch, and time series modeling and dimension regularization are sequentially performed to obtain a time series branch feature vector sequence corresponding to the time window one by one, wherein the sequential time series modeling and dimension regularization means that the time-dependent features of the electrical signal are extracted by using a time series convolution network, and the output dimension is unified by linear mapping and compression operation to form the corresponding time series branch feature vector sequence;

[0073] The aligned video frame or frame group is input into the lightweight image branch, and convolution processing and down-sampling are sequentially performed to obtain an image branch feature vector sequence corresponding to the time window one by one, wherein the sequential convolution processing and down-sampling means that the local structure features of the video image are extracted in the spatial dimension by a lightweight convolution network, and the feature dimension and redundant information are reduced by down-sampling operation after multi-layer convolution to form the image branch feature vector sequence corresponding to the time window;

[0074] The feature vector sequences of the time-frequency branch, the time sequence branch and the image branch are spliced in the channel dimension in the order of the time window, and the spliced feature vector sequences are weighted and fused based on the cross-modal interaction attention to generate a joint feature vector sequence, wherein the weighted and fused spliced feature vector sequences based on the cross-modal interaction attention means that the correlation between different modal features is established, the correlation weight of each modal feature at a time step is calculated, and the spliced feature vector sequences are combined by weight.

[0075] In the embodiment, the joint feature vector sequence is input into the phase change layered decoder, a layered recognition result is output by level, a phase change stage time window is determined, and an opening degree curve is output, comprising:

[0076] A phase change layered decoder is constructed, which is composed of a first layer state decoding layer, a second layer stage decoding layer and a third layer fault decoding layer, and the joint feature vector sequence is input into the phase change layered decoder.

[0077] In the first layer state decoding layer, the joint feature vector sequence is decoded in time sequence, and an on-off state sequence is output, wherein the on-off state includes opening, closing and transition.

[0078] In the second layer stage decoding layer, based on the preset phase sequence mask, only information transmission between the start-up stage and the acceleration stage, the acceleration stage and the stable stage, the stable stage and the deceleration stage, and the deceleration stage and the limit contact stage is allowed, and a monotonic gating rule is combined with the on-off state sequence and the actuator control quantity signal to output a phase change stage sequence and determine a phase change stage time window, wherein the preset phase sequence mask means that:

[0079] According to the physical order of the valve opening and closing process, all phase change stages are arranged in a fixed order of start-up, acceleration, stable, deceleration and limit contact, and a mask matrix is constructed which only allows connection between adjacent stages.

[0080] In the mask matrix, the connection weight of adjacent stages is retained, and the connection weight of non-adjacent stages is set to zero, so that only information transmission between reasonable adjacent stages is allowed in the decoding process, and unreasonable stage transitions such as jumping or reversing are avoided.

[0081] In the third layer fault decoding layer, context routing is performed between the on-off state sequence and the phase change stage sequence in the stage-specific fault path, and a fault preliminary judgment label sequence corresponding to each phase change stage is output, wherein the fault preliminary judgment label sequence includes stuck, leakage, limit failure, actuator anomaly and normal.

[0082] The opening degree regression head built in the phase change hierarchical decoder outputs the opening degree curve aligned with the phase change stage time window under the joint constraint of the opening and closing state sequence and the phase change stage sequence, and the opening and closing state sequence, the phase change stage sequence and the fault preliminary judgment label sequence are collectively referred to as the hierarchical recognition result.

[0083] The present application decomposes the valve opening and closing process into three levels of state, stage and fault for decoding by constructing a phase change hierarchical decoder, introduces a preset phase sequence mask and a monotonic gating rule to ensure the reasonable order and physical consistency of the phase change stage in the decoding process, and avoids the stage jump or reverse transfer problem that is prone to occur in the traditional method. Through the context routing mechanism, specific fault judgment paths can be selected in different phase change stages to realize the corresponding association of fault types and stage states, thereby improving the accuracy and interpretability of fault diagnosis. The built-in opening degree regression head outputs the opening degree curve under the dual constraint of state and stage results, which greatly reduces the deviation from the actual measurement, not only enhances the refinement degree of recognition and diagnosis, but also realizes the effective capture of early abnormalities and the unified output of multi-dimensional results, thereby improving the reliability and practicality of valve state monitoring.

[0084] In the embodiment, the opening degree regression head refers to a regression submodule arranged in the phase change hierarchical decoder for outputting the valve opening degree curve with time, the regression submodule takes the joint feature vector sequence, the opening and closing state sequence and the phase change stage sequence and the actuator control signal as input, performs time alignment according to the phase change stage time window, applies monotonic non-decreasing gating in the opening stage, applies monotonic non-increasing gating in the closing stage, and performs boundary clamping in the limit contact stage, and outputs the opening degree curve with consistent sampling period and value limited in the range of 0% to 100%.

[0085] In the embodiment, the output event modeling result includes:

[0086] The received phase change stage sequence and phase change stage time window, and the output joint feature vector sequence are received.

[0087] For each phase change stage time window, a feature subsequence aligned with the phase change stage time window is intercepted from the joint feature vector sequence, and the event type is determined as a limit impact event, a stuck micro-pulse event and a leakage voiceprint cluster event.

[0088] A phase change conditional point process layer is constructed based on the feature subsequence, and the phase change conditional point process layer includes a stage-specific trigger library, event mutual inhibition and cooperative gating, cross-modal confidence weighting and multi-resolution time grid, wherein:

[0089] The stage-specific trigger library corresponds to the start stage, the acceleration stage, the stable stage, the deceleration stage and the limit contact stage, respectively.

[0090] The event mutual inhibition and synergistic gating restricts the triggering relationship of different event types in the same time window, suppresses unreasonable concurrent triggering through mutual inhibition mechanism, and strengthens joint triggering with causal correlation through synergistic mechanism;

[0091] The cross-modal confidence weighting weights the event score according to the feature confidence of each branch;

[0092] The multi-resolution time grid simultaneously represents event triggering in fine-grained and coarse-grained time scales;

[0093] In each phase transition time window, a neural Hawkes point process is used to model the arrival time and occurrence intensity of the limit impact event, the stuck micro-pulse event and the leakage soundprint cluster event, to generate a candidate event timestamp set, output an event intensity curve changing with time, and calculate an event count expectation and an event type posterior distribution, and the neural Hawkes point process is used to model the arrival time and occurrence intensity of the limit impact event, the stuck micro-pulse event and the leakage soundprint cluster event, specifically as follows:

[0094] For the limit impact event, the acoustic pulse feature and the vibration impact feature when the valve approaches the limit position are used as the triggering input, and the neural Hawkes point process conditionally models the arrival intensity of the current time step according to the triggering of the previous event, to depict the concentrated burst characteristics and boundary alignment rules of the limit impact event;

[0095] For the stuck micro-pulse event, abnormal micro-fluctuations of the actuator control quantity and electrical signals are used as the triggering input, and the neural Hawkes point process captures the cumulative effect of multiple weak triggers in a short time scale, models the influence of historical events on the intensity of the current time step, and reveals the high-frequency small-amplitude pulse mode of the stuck event;

[0096] For the leakage soundprint cluster event, continuous frequency energy bands in acoustic signals and local morphological changes in video signals are used as the triggering input, and the neural Hawkes point process models the dependence of adjacent events in a longer time scale, generates a triggering intensity curve that decays or enhances with time, and represents the cluster-type triggering characteristics of the leakage event;

[0097] The calculation of the event count expectation and the event type posterior distribution is specifically as follows:

[0098] In each phase transition time window, the event intensity curve generated by the neural Hawkes point process is integrated in the time dimension to obtain the count expectation value of each type of event in the phase transition time window, which is used to represent the occurrence frequency of the event in a statistical sense;

[0099] On the basis of the candidate event timestamp set, the intensity values of different event types are normalized, the probability distribution of each event type at each time step is calculated, the posterior distribution of the event type is obtained, and the different event categories in the same time window are distinguished;

[0100] Perform stage boundary alignment and limit calibration, constrain and match the candidate event timestamp set with the boundary of the phase change stage time window, eliminate the candidate event timestamp falling outside the boundary, and determine the real event time at the boundary through the alignment calibration of the limit impact event, screen the effective trigger point by combining the confidence score of the stuck micro-pulse event and the leakage voiceprint cluster event, and obtain the event timestamp set;

[0101] The event timestamp set, the event intensity curve, the event count expectation and the event type posterior distribution of each phase change stage time window are summarized as the event modeling result.

[0102] The present application realizes high-precision capture and statistical modeling of transient abnormal events in the valve opening and closing phase change stage by introducing the phase change condition point process layer and the neural Hawkes point process in the event modeling process. The stage-specific trigger kernel library ensures the targeted extraction of event characteristics corresponding to different phase change stages, and the event mutual inhibition and cooperative gating effectively restricts unreasonable concurrent triggering and highlights causal related events. Cross-modal confidence weighting and multi-resolution time grid improve the robustness and multi-scale adaptability of event representation. The neural Hawkes point process further conditionally models the arrival time and intensity of limit impact, stuck micro-pulse and leakage voiceprint cluster events in the time dimension, which not only describes their characteristics of concentrated outbreak, cumulative triggering and cluster, but also outputs event count expectation and type posterior distribution, realizing quantitative expression of event frequency and category probability. This design breaks through the limitation of existing technology which can only rely on single signal or threshold detection, improves the sensitivity and diagnostic interpretation of early weak anomalies, and realizes fine monitoring and intelligent diagnosis of valve operation state.

[0103] In the embodiment, the output consistency checking result comprises:

[0104] Receive the phase change stage sequence, the phase change stage time window, the opening curve, the actuator control signal and the multi-modal time sequence sample, and perform time alignment according to the phase change stage time window;

[0105] A conditional reversible generation diagnostic device is constructed, which is composed of a conditional encoding unit, a reversible mapping unit and a consistency evaluation unit, wherein:

[0106] The reversible mapping unit is composed of a series of multi-level reversible coupled mapping layers, each level contains a scale transformation sublayer and a translation transformation sublayer and maintains the consistency of input and output dimensions;

[0107] The consistency evaluation unit is composed of a log-likelihood evaluation module and a residual aggregation module, the log-likelihood evaluation module outputs a conditional log-likelihood sequence at each time step, and the residual aggregation module outputs a modal-level reconstruction residual spectrum;

[0108] In each phase change stage time window, the phase change stage sequence, the phase identifier corresponding to the phase change stage time window, the opening curve and the actuator control quantity signal are combined into a conditional vector sequence by the conditional coding unit according to time steps, and length normalization and channel alignment are completed;

[0109] For each time step, the multi-modal time sequence sample and the conditional vector sequence are input into the reversible mapping unit to perform forward mapping to obtain an internal representation, and inverse mapping is performed under the same conditional vector to obtain a reconstruction segment, forming a reconstruction sequence corresponding to the time step, and the forward mapping to obtain an internal representation and the inverse mapping under the same conditional vector are specifically as follows:

[0110] In the forward mapping process, the multi-modal time sequence sample input is transformed layer by layer under the constraint of the conditional vector through coupled scale transformation and translation transformation, and high-dimensional observation data is mapped into low-dimensional reversible internal representation;

[0111] In the inverse mapping process, the internal representation is restored layer by layer to the reconstruction segment in the original data space in the reverse transformation order by using the same conditional vector as the constraint, so as to ensure the consistency between the input and the reconstruction;

[0112] The reconstruction sequence and the original sequence are calculated by the consistency evaluation unit at each time step, and the time-domain residual and the frequency-domain residual of the acoustic and vibration channels, the amplitude residual and the morphology residual of the electrical channel, and the spatial residual and the time sequence residual of the video channel are calculated, and are summarized into a modal-level reconstruction residual spectrum, and a conditional log-likelihood sequence corresponding to the time step is output, and the conditional log-likelihood sequence and the modal-level reconstruction residual spectrum are collectively referred to as consistency checking results, and the time-domain residual and the frequency-domain residual of the acoustic and vibration channels, the amplitude residual and the morphology residual of the electrical channel, and the spatial residual and the time sequence residual of the video channel are calculated, and are specifically as follows:

[0113] The time-domain residual of the acoustic and vibration channels is obtained by comparing the amplitude difference of the original waveform and the reconstructed waveform at each time step, and the frequency-domain residual is obtained by calculating the frequency energy distribution difference after the original waveform and the reconstructed waveform are respectively subjected to time-frequency transformation;

[0114] The amplitude residual of the electrical channel is obtained by comparing the current and voltage amplitude difference of the original signal and the reconstructed signal at the same time step, and the morphology residual is obtained by extracting the change trend of the original signal and the reconstructed signal in the time window and comparing the difference;

[0115] The spatial residual of the video channel is obtained by comparing the spatial distribution difference of the original image and the reconstructed image at the pixel or feature level frame by frame, and the temporal residual is obtained by analyzing the dynamic consistency of the video frame group at adjacent time steps and calculating the difference.

[0116] The present application realizes the reversible mapping and reconstruction of the multi-modal time sequence sample by constructing the conditional reversible generator, introducing the phase sequence, opening curve and actuator control signal into the conditional constraint, thereby establishing the bidirectional consistency between the input and output in the diagnosis process. Through the reversible mapping unit, the internal representation and reconstruction result can be obtained simultaneously under the constraint of the conditional vector, not only maintaining the integrity of the feature transformation, but also avoiding information loss. Through the consistency evaluation unit, combined with the double checking mechanism of conditional log-likelihood and modal level residual spectrum, the consistency and deviation between the diagnosis result and the original observation can be accurately quantified, realizing multi-level verification from global statistics to modal details. This design breaks through the limitations of traditional "black box" model, which is not interpretable and lacks result checking, making the valve opening and closing state and fault diagnosis result have higher credibility and interpretability, improving the sensitivity of early abnormal detection and the practicality of industrial application.

[0117] In the embodiment, the layered identification result, event modeling result and consistency checking result are fused to obtain the valve opening and closing state, opening, phase, fault type and severity score, including:

[0118] The layered identification result, event modeling result and consistency checking result are time-aligned according to the phase time window under the same time reference, resampled according to a unified sampling period, and the layered identification confidence, event arrival rate and event count expectation, log-likelihood and modal level residual are interval standardized;

[0119] In each phase, the layered identification result is mapped to state evidence, the event modeling result is mapped to event evidence, and the consistency checking result is mapped to consistency evidence according to the preset weight table and phase gating rule, and the three types of evidence are weighted and synthesized to generate phase-level fusion score and modal-level fusion score;

[0120] The valve opening and closing state and phase are determined according to the phase-level fusion score and modal-level fusion score, the fault type is determined, the opening is obtained combined with the opening curve, and the severity score is calculated, and the calculation of the severity score is specifically:

[0121] In each phase time window, the phase-level fusion score and modal-level fusion score are taken as the basic risk quantity, combined with the event intensity, count expectation and type posterior in the event modeling result, and the preset weight and penalty coefficient are applied to the leakage, sticking and limit impact respectively to obtain the normalized intra-phase risk value;

[0122] The risk value in each stage is corrected according to the consistency checking result:

[0123] The amplitude is adjusted upwards when the log-likelihood is low or the modal level residual spectrum increases, and the amplitude is adjusted downwards when both are in the normal interval;

[0124] The severity score of each stage is obtained by length weighting and criticality weighting according to the duration of each stage;

[0125] The severity scores of each stage are summarized according to the stage weight and time sequence to form the overall severity benchmark score, and the confidence is reduced or increased according to the confidence of the hierarchical identification result, and finally the historical calibration table is mapped to the final severity score and corresponding grade for alarm triggering and linkage control, wherein the historical calibration table refers to a mapping reference table established based on a large amount of valve operation and maintenance data, which statistically calibrates the severity levels corresponding to different event intensities, residual levels, log-likelihood intervals and actual maintenance results, forming a corresponding relationship between risk values and severity scores.

[0126] Embodiment 1

[0127] In order to verify the feasibility of the application in implementation, the application is applied to a valve opening and closing state monitoring system of a coastal petrochemical enterprise refinery, the device scale of the plant is 35000 tons of crude oil per day, the key process links include hydrocracking, atmospheric and vacuum distillation, reforming and coking. There are a total of 1432 electric valves, pneumatic valves and hydraulic valves in the whole plant, among which about 156 main valves of the key production line are frequently operated and bear the core task of adjusting raw material flow and controlling temperature and pressure. Once the valve opening and closing is not timely or there is jamming and leakage, it will directly affect the production safety and economic benefits.

[0128] In the past, the plant mainly used a single current sensor or motor current curve method to determine the opening and closing state of the valve. In actual operation, threshold drift and misjudgment often occur, for example, under high temperature working conditions, the motor load fluctuates greatly, which easily causes valve jamming misjudgment; under low temperature and high humidity working conditions, the acoustic signal attenuation is obvious, which leads to insensitive leakage detection. According to the maintenance records of the plant, there were 22 production interruptions caused by valve state misjudgment in 2024, with an average of 3.5 hours of downtime each time, and direct economic losses of about 4.7 million yuan.

[0129] After the introduction of the valve opening and closing state recognition and diagnosis method based on deep learning proposed in the application, it is first deployed on the key reforming device. The monitoring object is 20 main electric regulating valves, each of which is equipped with acoustic, vibration, electrical and video acquisition devices, and is connected with the actuator control signal to the monitoring system proposed in the application. The system realizes millisecond-level synchronization with a unified time reference, and performs denoising, segmentation and normalization processing on the collected signals. After time-frequency transformation, the acoustic and vibration signals generate time-frequency spectrograms, and the electrical signals and video signals are standardized in amplitude and shape to form multi-modal time series samples.

[0130] Subsequently, the system models the time-frequency spectrograms, electrical segments and video frames through a cross-modal feature extraction network, fuses to generate a joint feature vector sequence, and inputs into a phase change hierarchical decoder. The decoder outputs the opening and closing state sequence, the phase change stage sequence and the fault preliminary judgment result in turn, and generates an opening degree curve aligned with the phase change stage window by the built-in opening degree regression head. In a valve operation from full closing to full opening, the system accurately distinguishes five stages: starting, accelerating, stabilizing, decelerating and limit contact, and the average deviation of the opening degree curve from the field manual measurement result is not more than 1.8%.

[0131] In terms of anomaly detection, the application uses neural Hawkes point process to model the limit impact, stuck micro-pulse and leakage sound cluster. In a reforming reactor switching process in March 2025, the system accurately captures the stuck micro-pulse event at 3.4 seconds of the valve, and detects a slight leakage sound cluster in the limit contact stage. The field repair result shows that the valve packing gland is loose, and the leakage amount is about 0.3 L / min. The system alarm is highly consistent with the repair result, proving that the application has high sensitivity to early hidden faults.

[0132] To verify the effectiveness of the application, the plant conducted a comparative test from March to June 2025, and counted the state recognition and fault detection performance of 20 valves in daily opening and closing.

[0133] Table 1 Comparison of valve state recognition and fault detection data

[0134] Index category Conventional method value Invention method value Promotion effect State recognition accuracy 87.2% 96.8% +9.6% Stuck event detection rate 59.7% 91.6% +31.9% Leakage event detection rate 43.5% 88.3% +44.8% False alarm rate 12.4% 3.1% -9.3% Average alarm confirmation rate 62.8% 88.2% +25.4% Opening curve deviation mean value 6.4% 1.8% -4.6%

[0135] As can be seen from the data in Table 1, the application has a significant improvement in valve opening and closing state recognition and fault diagnosis compared with the traditional method. In terms of overall state recognition accuracy, the traditional method can only reach 87.2%, while the method of the application reaches 96.8%, an increase of 9.6 percentage points, indicating that the application can more comprehensively and accurately capture the feature information in the valve operation process through multi-modal feature fusion and hierarchical decoding mechanism.

[0136] In terms of abnormal event detection, the advantages of the present application are more obvious. For the stuck event, the detection rate of the traditional method is less than 60%, only 59.7%, while the detection rate of the present application is as high as 91.6%, an increase of 31.9 percentage points, which reflects the high sensitivity of neural Hawkes point process to transient abnormal events. For the leakage event, the detection rate of the traditional method is only 43.5%, while the present application is improved to 88.3%, an increase of 44.8 percentage points, which shows that through the joint analysis of acoustic, vibration and video signals, the present application can effectively identify the hidden leakage that is difficult to be found by the traditional method.

[0137] In terms of system reliability, the present application also has significant improvement. The false positive rate of the traditional method is as high as 12.4%, which is easy to cause unnecessary maintenance and shutdown, while the false positive rate of the present application is only 3.1%, a decrease of 9.3 percentage points, which shows that the consistency checking mechanism greatly improves the credibility of the diagnosis result. At the same time, the average alarm confirmation rate is increased from 62.8% to 88.2%, an increase of 25.4 percentage points, which shows that the alarm issued by the system is more reliable and can be effectively verified by the on-site maintenance result. Finally, in terms of the accuracy of the opening curve, the traditional method has an average deviation of 6.4% from the actual measurement result, while the present application reduces the deviation to 1.8%, which shows that the synergistic effect of the opening regression head and the layered decoder significantly improves the quantization accuracy in the valve opening and closing process.

[0138] The present application not only has better identification accuracy than the traditional method, but also has breakthrough improvement in key indicators such as abnormal event detection, alarm reliability and opening curve accuracy, which fully proves the feasibility and practical value of the present application under complex working conditions.

[0139] The above is only the preferred specific embodiment of the present application, but the protection scope of the present application is not limited thereto, any skilled person in the art can make equivalent replacement or change according to the technical solution and the inventive concept of the present application within the technical range disclosed by the present application, which should be covered within the protection scope of the present application.

Claims

1. A valve opening and closing state recognition and diagnosis method based on deep learning, characterized in that, The method comprises the following steps: Collecting acoustic signals, vibration signals, electrical signals, video signals and actuator control signals during valve operation, synchronizing at the millisecond level based on a unified time reference, performing denoising, segmentation and normalization processing, performing time-frequency transformation on the acoustic signals and vibration signals to obtain time-frequency spectrograms, and performing amplitude and shape normalization on the electrical signals and video signals to form multi-modal time series samples; Input the time-frequency spectrograms into the time-frequency convolution branch, input the electrical signals into the time series modeling branch, and input the video signals into the lightweight image branch to extract branch features respectively, and generate a joint feature vector sequence through cross-modal interaction attention fusion; Input the joint feature vector sequence into a phase change hierarchical decoder, output hierarchical recognition results by level, determine a phase change stage time window, and output an opening degree curve; Within the phase change stage time window, construct a phase change conditional point process layer based on the joint feature vector sequence, model the arrival time and occurrence intensity of limit impact events, stuck micro-pulse events and leakage soundprint cluster events using a neural Hawkes point process, and output event modeling results; Construct a conditional reversible generation diagnostic device using the phase change stage sequence and the opening degree curve, as well as the actuator control signal to form a condition set, perform conditional generation and reversible reconstruction on the multi-modal time series samples, and output consistency verification results; Fuse the hierarchical recognition results, event modeling results and consistency verification results to obtain valve opening and closing states, opening degrees, phase change stages, fault types and severity scores, trigger an alarm and issue a linkage control instruction when the scores exceed a preset threshold, and perform drift monitoring and recalibration and archive the diagnostic results; The method of inputting the joint feature vector sequence into the phase change hierarchical decoder, outputting hierarchical recognition results by level, determining a phase change stage time window, and outputting an opening degree curve comprises the following steps: Construct a phase change hierarchical decoder composed of a first layer state decoding layer, a second layer stage decoding layer and a third layer fault decoding layer, and input the joint feature vector sequence into the phase change hierarchical decoder; In the first layer state decoding layer, decode the joint feature vector sequence in chronological order to output an opening and closing state sequence; In the second layer stage decoding layer, based on a preset phase order mask, only allow information transmission between the start-up stage and the acceleration stage, the acceleration stage and the stable stage, the stable stage and the deceleration stage, and the deceleration stage and the limit contact stage, and combine the opening and closing state sequence and the actuator control signal to apply a monotonic gating rule to output a phase change stage sequence and determine a phase change stage time window; In the third layer fault decoding layer, according to the opening and closing state sequence and the phase change stage sequence, route the context between the stage-specific fault paths to output a fault preliminary judgment label sequence corresponding to each phase change stage; Under the joint constraints of the opening and closing state sequence and the phase change stage sequence, output an opening degree curve aligned with the phase change stage time window by the opening degree regression head built in the phase change hierarchical decoder, and collectively refer to the opening and closing state sequence, the phase change stage sequence and the fault preliminary judgment label sequence as hierarchical recognition results.

2. The valve opening and closing state recognition and diagnosis method based on deep learning according to claim 1, characterized in that, The millisecond-level synchronization based on the unified time reference, the denoising, the segmentation and the normalization processing refer to time alignment of the acoustic signal, the vibration signal, the electrical signal, the video signal and the actuator control quantity signal according to the same time reference, removal of high-frequency noise and direct current components by using filtering processing, division of the continuous signal into multiple time segments according to a preset sampling length, and unification of different modal signals to a comparable scale by amplitude standardization and mean-variance normalization. 3.The valve opening and closing state recognition and diagnosis method based on deep learning according to claim 1, characterized in that, The time-frequency transformation of the acoustic signal and the vibration signal to obtain a time-frequency spectrum, and the amplitude and morphology standardization of the electrical signal and the video signal refer to frequency distribution characteristics obtained by the transformation from the time domain to the frequency domain within a fixed time window, and formation of a two-dimensional time-frequency spectrum, amplitude normalization of the electrical signal to eliminate amplitude differences of different measurement channels, and morphology standardization processing of the video signal by size scaling and gray scale distribution equalization.

4. The valve opening and closing state recognition and diagnosis method based on deep learning according to claim 1, characterized in that, The joint feature vector sequence generated by the cross-modal interaction attention fusion includes: From the formed multi-modal time sequence sample, the time-frequency spectrum of the acoustic and vibration signals, the time segment of the electrical signal, and the video frame or frame group aligned with the time segment are obtained according to the unified time window; The time-frequency spectrum is input into the time-frequency convolution branch in channel order, and convolution processing and feature compression are sequentially performed to obtain a time-frequency branch feature vector sequence corresponding to the time window; The time segment of the electrical signal is input into the time sequence modeling branch, and time sequence modeling and dimension regularization are sequentially performed to obtain a time sequence branch feature vector sequence corresponding to the time window; The aligned video frame or frame group is input into the lightweight image branch, and convolution processing and down-sampling are sequentially performed to obtain an image branch feature vector sequence corresponding to the time window; The feature vector sequences of the time-frequency branch, the time sequence branch and the image branch are spliced in the channel dimension in the time window order, the spliced feature vector sequence is weighted and fused based on the cross-modal interaction attention, and a joint feature vector sequence is generated.

5. The valve opening and closing state recognition and diagnosis method based on deep learning according to claim 1, characterized in that, The opening degree regression head refers to a regression submodule arranged in the phase change layered decoder for outputting a valve opening degree curve changing with time. The regression submodule takes the joint feature vector sequence, the opening and closing state sequence, the phase change stage sequence and the actuator control quantity signal as input, performs time alignment according to the phase change stage time window, applies a monotone non-decreasing gate in the opening stage, applies a monotone non-increasing gate in the closing stage, and performs boundary clamping in the limit contact stage. An opening degree curve with consistent sampling period and value limited in the range of 0% to 100% is output.

6. The valve opening and closing state recognition and diagnosis method based on deep learning according to claim 1, characterized in that, The output event modeling result includes: The received phase change stage sequence and phase change stage time window, and the output joint feature vector sequence; For each phase change stage time window, a feature sub-sequence aligned with the phase change stage time window is intercepted from the joint feature vector sequence, and the event type is determined as a limit impact event, a stuck micro-pulse event and a leakage voiceprint cluster event. A phase change condition point process layer is constructed based on a feature subsequence, and the phase change condition point process layer includes a stage-specific trigger core library, event mutual inhibition and synergistic gating, cross-modal confidence weighting, and a multi-resolution time grid, wherein: The stage-specific trigger core library corresponds to a start stage, an acceleration stage, a stable stage, a deceleration stage, and a limit contact stage, respectively. The event mutual inhibition and synergistic gating constrains the trigger relationship of different event types within the same time window, suppresses unreasonable concurrent triggers through a mutual exclusion mechanism, and strengthens joint triggers with causal correlations through a synergistic mechanism. The cross-modal confidence weighting weights the event score according to the feature confidence of each branch. The multi-resolution time grid simultaneously represents event triggers at fine-grained and coarse-grained time scales. Within each phase change stage time window, a neural Hawkes point process is used to model the arrival time and intensity of limit impact events, stuck micro-pulse events, and leakage voiceprint cluster events, generate a candidate event timestamp set, output an event intensity curve that changes over time, and calculate the event count expectation and event type posterior distribution. The stage boundary alignment and limit calibration are performed to constrain and match the candidate event timestamp set with the boundaries of the phase change stage time window, eliminate candidate event timestamps that fall outside the boundaries, and determine the real event time at the boundary through the alignment and calibration of limit impact events. Combined with the confidence score of stuck micro-pulse events and leakage voiceprint cluster events, the effective trigger points are selected to obtain the event timestamp set. The event timestamp set, event intensity curve, event count expectation, and event type posterior distribution of each phase change stage time window are summarized as event modeling results.

7. The valve opening and closing state recognition and diagnosis method based on deep learning according to claim 1, characterized in that, The output consistency verification result includes: Receiving a phase change stage sequence, a phase change stage time window, an opening curve, an actuator control signal, and a multi-modal time sequence sample, and performing time alignment according to the phase change stage time window; A conditionally reversible generation diagnostic device is constructed, which consists of a conditional encoding unit, a reversible mapping unit, and a consistency evaluation unit, wherein: The reversible mapping unit is composed of a series of multi-level reversible coupled mapping layers, each level containing a scale transformation sublayer and a translation transformation sublayer and maintaining consistent input and output dimensions. The consistency evaluation unit consists of a log-likelihood evaluation module and a residual aggregation module. The log-likelihood evaluation module outputs a sequence of conditional log-likelihoods at each time step, and the residual aggregation module outputs a modal-level reconstruction residual spectrum. Within each phase change stage time window, the conditional encoding unit combines the phase change stage sequence, the stage identifier corresponding to the phase change stage time window, the opening curve, and the actuator control signal into a conditional vector sequence according to the time step, and completes length normalization and channel alignment; For each time step, the multi-modal time sequence sample and the conditional vector sequence are input into the reversible mapping unit to perform forward mapping to obtain an internal representation, and inverse mapping is performed under the same conditional vector to obtain a reconstruction fragment, forming a reconstruction sequence corresponding to the time step. The consistency evaluation unit calculates the time-domain and frequency-domain residual errors of the acoustic and vibration channels, the amplitude and morphology residual errors of the electrical channel, and the spatial and timing residual errors of the video channel for each time step of the reconstructed sequence and the original sequence, and aggregates them into a modal-level reconstruction residual spectrum. The conditional log-likelihood sequence corresponding to the time step is also output. The conditional log-likelihood sequence and the modal-level reconstruction residual spectrum are collectively referred to as consistency check results.

8. The valve opening and closing state recognition and diagnosis method based on deep learning according to claim 1, characterized in that, The fusion of the hierarchical identification results, event modeling results, and consistency check results obtains the valve opening and closing state, opening degree, phase change stage, fault type, and severity score, including: The hierarchical identification results, event modeling results, and consistency check results are time-aligned according to the phase change stage time window and resampled according to a unified sampling period under the same time reference. The hierarchical identification confidence, event arrival rate, and event count expectation, log-likelihood, and modal-level residual are interval standardized. In each phase change stage, the hierarchical identification results are mapped to state evidence, the event modeling results are mapped to event evidence, and the consistency check results are mapped to consistency evidence according to the preset weight table and stage gating rules. The three types of evidence are weighted and combined to generate stage-level fusion scores and modal-level fusion scores. The stage-level fusion scores and modal-level fusion scores are used to determine the valve opening and closing state and the phase change stage, determine the fault type, obtain the opening degree based on the opening degree curve, and calculate the severity score.

Images