Household gas leakage discrimination method based on multi-modal fusion and deep learning

Abstract

This invention discloses a method for identifying gas leaks in residential homes based on multimodal fusion and deep learning. To address the problems of pressure transient fluctuations and concentration perturbations caused by normal gas usage behavior, as well as pressure background fluctuations due to pipeline pressure regulation and neighboring households' gas usage, which make single-threshold judgment prone to false alarms and missed alarms, and difficult to identify minute leaks and slow seepage in a timely manner, this invention acquires indoor combustible gas concentration, terminal gas supply pressure, and gas flow signals and performs time-aligned preprocessing. It then constructs a neural state-space model based on pressure and flow, uses neural Kalman filtering to estimate the pressure background component, performs background stripping on the pressure signal to obtain the pressure residual, and further performs time window segmentation and multimodal feature fusion on the pressure residual, concentration, and flow. The residual is then input into a residual leak discrimination network to output a leak score and discrimination result. Combining the score time accumulation and threshold judgment, an early warning is output, achieving accurate differentiation of real leaks under complex gas usage disturbances and stable, low-false-alarm early warning technology for minute leaks and slow seepage.

Description

Technical Field

[0001] The present invention relates to the field of gas safety monitoring and leakage warning, and particularly to a method for discriminating household gas leakage based on multimodal fusion and deep learning. Background Art

[0002] With the popularization of urban household gas and the development of Internet of Things monitoring technology, household gas safety monitoring has gradually evolved from traditional manual inspection to online and intelligent directions. Existing household monitoring devices usually configure combustible gas concentration sensors, and can collect gas-related signals in combination with intelligent gas meters, end pressure sensors, and flow meters, and achieve remote alarm and linkage control through a gateway or cloud platform. In terms of discrimination methods, concentration threshold alarm, pressure threshold alarm, or rule judgment based on pressure fluctuation characteristics and flow changes are mostly used in engineering applications; in recent years, machine learning models trained based on historical data have also emerged to identify and classify abnormal gas usage states in order to reduce false alarms and improve alarm timeliness.

[0003] However, the above existing technologies still have deficiencies in the actual scenarios of household gas monitoring, mainly reflected in:

[0004] 1. Existing methods mostly rely on single signals or fixed thresholds and rules. Normal gas usage behaviors such as residents' ignition, valve opening and closing, and cooking will cause transient fluctuations in end pressure and slight changes in indoor concentration, which are likely to lead to false alarms; at the same time, background pressure fluctuations caused by neighboring household gas usage, pipe network pressure regulation, etc. will cover up abnormal features and cause missed alarms.

[0005] 2. For early risks such as minor leaks and slow leaks, the concentration increase is small and is easily affected by ventilation, noise, and sensor drift, and the pressure change may also be submerged by background fluctuations. Existing determination methods are difficult to stably and timely identify.

[0006] 3. Existing multi-source information joint analysis means are insufficient, lacking an effective modeling and stripping mechanism for pressure background components, resulting in limited robustness and generalization ability of multimodal fusion discrimination under complex disturbances.

[0007] Therefore, a method for discriminating household gas leakage that can solve the deficiencies of the above existing technologies is needed. Summary of the Invention

[0008] One objective of this invention is to propose a method for detecting gas leaks in residential homes based on multimodal fusion and deep learning. Addressing the shortcomings of existing technologies that rely solely on single concentration or pressure thresholds for gas leak detection, which are susceptible to disturbances from normal gas usage such as ignition, valve switching, and cooking, as well as pressure fluctuations caused by pipeline pressure regulation and neighboring households' gas usage, leading to false alarms and missed alarms, and making it difficult to promptly identify early risks of minor leaks and slow seepage, the following technical solution is proposed: Acquiring indoor combustible gas concentration, terminal gas supply pressure, and gas flow signals, and performing time alignment and preprocessing; constructing a neural state-space model based on pressure and flow, and recursively estimating the pressure background component using neural Kalman filtering; performing background stripping on the pressure signal to obtain the pressure residual; segmenting the pressure residual, concentration, and flow rate into time windows and fusing multimodal features, inputting the residual into a leak detection network to output a leak score and detection result, and accumulating the leak score over time to output a warning result. This invention has the technical effect of accurately distinguishing between normal gas usage and actual leakage under complex normal gas usage disturbances and pressure background fluctuations, and achieving stable early warning with low false alarms for minor leaks and slow seepage.

[0009] This invention provides a method for detecting in-home gas leaks based on multimodal fusion and deep learning, comprising:

[0010] S1. In a home gas monitoring scenario, acquire indoor combustible gas concentration signal sequences, terminal gas supply pressure signal sequences, and gas flow signal sequences, and preprocess them to achieve time alignment; S2. Construct a neural state-space model based on the terminal gas supply pressure signal sequences and gas flow signal sequences. The neural state-space model includes state variables representing the background component of the terminal gas supply pressure, a state transition function parameterized by a neural network, and an observation function parameterized by a neural network; S3. Perform neural Kalman filtering recursive estimation on the terminal gas supply pressure signal sequences using the neural state-space model to obtain the estimated sequence of the terminal gas supply pressure background component; S4. Based on the estimated sequence of the terminal gas supply pressure background component, perform on-site gas monitoring on the terminal gas supply pressure signal sequences. Background stripping is performed on the gas supply pressure signal sequence to obtain the pressure residual signal sequence, which is the difference sequence between the terminal gas supply pressure signal sequence and the estimated sequence of the terminal gas supply pressure background component; S5, the pressure residual signal sequence, indoor combustible gas concentration signal sequence, and gas flow signal sequence are segmented according to a preset time window, and multimodal feature fusion is performed within the time window to obtain a multimodal fused feature sequence; S6, the multimodal fused feature sequence is input into the residual leakage discrimination network, and the leakage score and discrimination result are output. The discrimination result is used to characterize the normal gas use state or the gas leakage state; S7, the leakage score is accumulated over time and compared with a preset judgment threshold. Combined with the discrimination result, the in-home gas leakage early warning result is output.

[0011] Optionally, S1 includes:

[0012] The sampling times for the indoor combustible gas concentration signal sequence, the terminal gas supply pressure signal sequence, and the gas flow signal sequence are obtained respectively, and the three are resampled based on a unified time axis.

[0013] When any signal sequence has missing sampling points on the unified time axis, interpolation is used to fill in the missing sampling points to complete time alignment;

[0014] After time alignment is completed, the indoor combustible gas concentration signal sequence, the terminal gas supply pressure signal sequence, and the gas flow signal sequence are low-pass filtered to remove high-frequency noise and complete the noise reduction process.

[0015] After denoising, the median and median absolute deviation of each signal sequence are calculated within a sliding window of a preset length. Sampling points that deviate from the median and whose deviation exceeds a preset multiple of the median absolute deviation are identified as outliers, and the outliers are replaced with the median within the sliding window to complete the outlier removal.

[0016] After outlier removal, the mean and standard deviation of each signal sequence are calculated within a sliding window of a preset length. The sampling points within the window are then processed by subtracting the mean and dividing by the standard deviation to complete the normalization process.

[0017] Optionally, S2 includes:

[0018] The state variables of the neural state space model include the end-supply pressure background state, which characterizes the end-supply pressure background component, and the change state, which characterizes the end-supply pressure background component over time.

[0019] The state transition function is parameterized by a neural network and is used to update the state variable at the current sampling time based on the state variable at the previous sampling time and the gas flow rate signal sequence at the current sampling time.

[0020] The observation function is parameterized by a neural network and is used to generate a predicted value of the terminal gas supply pressure based on the state variable at the current sampling time. The predicted value of the terminal gas supply pressure is associated with the observed value at the corresponding sampling time of the terminal gas supply pressure signal sequence, so that the neural state space model can be used to characterize the background component of the terminal gas supply pressure.

[0021] The initial state estimation is determined based on the statistics of the terminal gas supply pressure signal sequence in the initial time period, and the initial error covariance is determined based on a preset covariance parameter, wherein the preset covariance parameter is used to characterize the confidence level of the initial state estimation.

[0022] Optionally, S3 includes:

[0023] The neural Kalman filter recursive estimation uses the initial state estimation and the initial error covariance as initial conditions, and performs prediction update and correction update sequentially for each sampling time of the terminal gas supply pressure signal sequence;

[0024] In the prediction update, based on the state transition function of the neural state space model and combined with the value of the gas flow signal sequence at the current sampling time, the state variable at the previous sampling time is predicted to obtain the predicted state variable at the current sampling time, and the prediction error covariance is updated based on the predicted state variable and the process noise covariance.

[0025] In the correction update, the predicted state variables are mapped to predicted values ​​of end-gas pressure based on the observation function of the neural state space model.

[0026] Based on the difference between the predicted value of the terminal gas supply pressure and the observed value of the terminal gas supply pressure signal sequence at the corresponding sampling time, the Kalman gain is calculated by combining the prediction error covariance and the observation noise covariance, and the predicted state variable is corrected by the Kalman gain to obtain the corrected state variable, and the correction error covariance is updated at the same time.

[0027] Based on the corrected state variables at each sampling time, output the estimated sequence of the background component of the terminal gas supply pressure;

[0028] Furthermore, at least one of the process noise covariance and the observation noise covariance is an adaptive covariance, which is output by a covariance estimation network.

[0029] The input to the covariance estimation network includes the difference between the predicted value of the terminal gas supply pressure and the observed value of the terminal gas supply pressure signal sequence, and at least one of the gas flow signal sequence.

[0030] Optionally, S4 includes:

[0031] The estimated sequence of background component of terminal gas supply pressure corresponds one-to-one with the signal sequence of terminal gas supply pressure at the same sampling time.

[0032] For any sampling time, the value of the terminal gas supply pressure signal sequence at that sampling time is subtracted from the value of the terminal gas supply pressure background component estimation sequence at that sampling time to obtain the pressure residual signal corresponding to that sampling time. Thus, the pressure residual signal sequence is composed of the pressure residual signals at each sampling time, wherein the pressure residual signal sequence and the terminal gas supply pressure signal sequence have the same set of sampling times.

[0033] Optionally, S5 includes:

[0034] The pressure residual signal sequence, indoor combustible gas concentration signal sequence, and gas flow signal sequence are synchronously slid segmented with a preset window length and a preset sliding step size, so that within any sliding time window, the pressure residual signal subsequence, indoor combustible gas concentration signal subsequence, and gas flow signal subsequence corresponding to that sliding time window are obtained respectively, and the pressure residual signal subsequence, the indoor combustible gas concentration signal subsequence, and the gas flow signal subsequence correspond one-to-one at the sampling time;

[0035] Within each sliding time window, features for characterizing temporal changes are extracted from the pressure residual signal subsequence, the indoor combustible gas concentration signal subsequence, and the gas flow rate signal subsequence. These features include mean, standard deviation, maximum value, minimum value, and first-order difference statistics. The features corresponding to the pressure residual signal subsequence, the indoor combustible gas concentration signal subsequence, and the gas flow rate signal subsequence are then concatenated in a preset order to obtain the multimodal fusion features corresponding to that sliding time window.

[0036] The multimodal fusion features corresponding to each sliding time window are arranged in chronological order to obtain the multimodal fusion feature sequence.

[0037] Optionally, S6 includes:

[0038] The residual leakage discrimination network includes a temporal feature extraction subnetwork and a discrimination subnetwork. The temporal feature extraction subnetwork is used to perform temporal modeling on the multimodal fusion feature sequence and output a temporal feature representation. The discrimination subnetwork is used to classify the temporal feature representation and output a leakage score.

[0039] The leak score is a probability value or confidence value that characterizes the state of a gas leak.

[0040] A judgment result is generated based on the comparison result between the leakage score and the preset classification threshold. When the leakage score is not less than the preset classification threshold, the judgment result is determined to be a gas leakage state.

[0041] When the leakage score is less than the preset classification threshold, the judgment result is determined to be a normal gas usage state;

[0042] Furthermore, the residual leakage discrimination network also outputs gas consumption behavior category results, which include at least one of ignition gas consumption, steady-state gas consumption, and valve shutdown.

[0043] Furthermore, when outputting a leakage score, the discriminative subnetwork uses the gas usage behavior category result as an auxiliary input or auxiliary task constraint to reduce false alarms caused by normal gas usage disturbances.

[0044] Optionally, the S7 includes:

[0045] The leakage scores are arranged in chronological order to form a leakage score sequence, and the leakage score sequence is accumulated over time based on a preset accumulation rule to obtain a cumulative leakage score. The preset accumulation rule includes at least one of sliding window accumulation and exponential weighted accumulation.

[0046] The cumulative leakage score is compared with a preset judgment threshold, and the number of times the judgment result is a gas leakage state within the preset judgment time window is counted.

[0047] When the cumulative leakage score is not less than the preset judgment threshold, and the number of times the judgment result is gas leakage state within the preset judgment time window is not less than the preset number threshold, the gas leakage warning result for entering the house is output as a warning state.

[0048] When the cumulative leakage score is less than the preset judgment threshold, or when the number of times the judgment result is a gas leakage state within the preset judgment time window is less than the preset number threshold, the gas leakage warning result for entering the house is output as a non-warning state.

[0049] The beneficial effects of this invention are:

[0050] 1. By constructing a neural state-space model based on pressure and flow rate and using neural Kalman filtering to recursively estimate the background component of the terminal gas supply pressure, and then performing background stripping on the pressure signal to obtain the pressure residual, the pressure background fluctuation interference caused by pipeline pressure regulation and neighboring gas consumption can be effectively suppressed, thereby improving the stability and availability of pressure anomaly characterization under complex operating conditions.

[0051] 2. The pressure residual, indoor combustible gas concentration signal and gas flow signal are synchronously segmented according to time window and multimodal feature fusion is performed. The leakage score and discrimination result are output through the residual leakage discrimination network. Compared with the judgment method based on a single concentration threshold or a single pressure threshold, it can more accurately distinguish between normal gas use disturbances such as ignition, valve switching and cooking and real leakage, and reduce false alarms and false alarms.

[0052] 3. By accumulating leakage scores over time and combining them with judgment and frequency thresholds for early warning output, continuous evidence accumulation can be used to make judgments on early risks such as minor leaks and slow seepage, thereby improving the robustness and consistency of early warnings and further reducing false triggering caused by occasional noise or short-term disturbances. Attached Figure Description

[0053] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0054] Figure 1This is a flowchart of a gas leak detection method for in-home gas leaks based on multimodal fusion and deep learning proposed in this invention. Detailed Implementation

[0055] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0056] refer to Figure 1 A method for detecting in-home gas leaks based on multimodal fusion and deep learning, comprising:

[0057] S1. In a home gas monitoring scenario, acquire indoor combustible gas concentration signal sequences, terminal gas supply pressure signal sequences, and gas flow signal sequences, and preprocess them to achieve time alignment; S2. Construct a neural state-space model based on the terminal gas supply pressure signal sequences and gas flow signal sequences. The neural state-space model includes state variables representing the background component of the terminal gas supply pressure, a state transition function parameterized by a neural network, and an observation function parameterized by a neural network; S3. Perform neural Kalman filtering recursive estimation on the terminal gas supply pressure signal sequences using the neural state-space model to obtain the estimated sequence of the terminal gas supply pressure background component; S4. Based on the estimated sequence of the terminal gas supply pressure background component, perform on-site gas monitoring on the terminal gas supply pressure signal sequences. Background stripping is performed on the gas supply pressure signal sequence to obtain the pressure residual signal sequence, which is the difference sequence between the terminal gas supply pressure signal sequence and the estimated sequence of the terminal gas supply pressure background component; S5, the pressure residual signal sequence, indoor combustible gas concentration signal sequence, and gas flow signal sequence are segmented according to a preset time window, and multimodal feature fusion is performed within the time window to obtain a multimodal fused feature sequence; S6, the multimodal fused feature sequence is input into the residual leakage discrimination network, and the leakage score and discrimination result are output. The discrimination result is used to characterize the normal gas use state or the gas leakage state; S7, the leakage score is accumulated over time and compared with a preset judgment threshold. Combined with the discrimination result, the in-home gas leakage early warning result is output.

[0058] In this specific embodiment, S1 includes:

[0059] Preprocessing uses a unified time axis to achieve time alignment of indoor combustible gas concentration signal sequence, terminal gas supply pressure signal sequence, and gas flow signal sequence, and completes noise reduction, outlier removal, and normalization processing.

[0060] The raw sequences with timestamps were obtained from the concentration sensor, pressure sensor, and flow meter, respectively. The indoor combustible gas concentration signal sequence is denoted as follows: The terminal gas supply pressure signal sequence is denoted as The gas flow signal sequence is denoted as ,in The three types of signals are respectively the first Each sampling time uses the same clock reference. The indoor combustible gas concentration sample value at the corresponding sampling time is expressed as follows: express, The terminal gas supply pressure sampled at the corresponding sampling time is expressed in kPa. The gas flow rate sampled at the corresponding sampling time and expressed as follows: express, , These represent the number of sampling points for the three types of original sequences, respectively.

[0061] Then construct a unified timeline:

[0062] ;

[0063] in This is the start time of the area jointly covered by the three types of signals. The resampling time interval. To unify the sampling point sequence numbers of the time axis and satisfy the following conditions The termination time of the area jointly covered by no more than three types of signals;

[0064] Next, resampling was performed on the three types of signals based on a unified time axis to obtain a time-aligned indoor combustible gas concentration signal sequence. Terminal gas supply pressure signal sequence With gas flow signal sequence ,in To standardize the number of sampling points on the time axis, resampling is implemented using linear interpolation. At any given moment on the standardized time axis... When there are no samples taken at the same time in the corresponding original sequence, the location satisfies... The two adjacent original sampling times are obtained by linear interpolation at two points. in for either of them and For the aligned signal at time The sampled value, when When a signal is located outside the beginning and end boundaries of its original sequence, the sampled values ​​at the boundary are preserved to fill in the missing points, thus ensuring that the three types of signals correspond point by point on a unified time axis.

[0065] After completing time alignment, Low-pass filtering is performed separately for noise reduction. The low-pass filter is a fourth-order Butterworth digital low-pass filter. Sampling frequency, with To determine the filter coefficients for the cutoff frequency, the filtering implementation employs forward-backward zero-phase filtering and mirror extension of the beginning and end of the sequence to suppress boundary transients, thereby obtaining three types of denoised sequences.

[0066] After denoising, each type of sequence is processed using a length of [length missing]. Outlier removal is performed using a sliding window with each sampling point. The window is centered on the current sampling point and padded with mirrors at both ends of the sequence to ensure that each sampling point has a complete window. Within each window, the median and the absolute deviation of the median of the sampled values ​​within that window are calculated and multiplied by a threshold. Outliers are identified when the deviation of a sampling point from the median of the window exceeds a certain threshold. When the absolute deviation of the median is doubled, the sampling point is identified as an outlier and replaced with the median of the window, thus obtaining the outlier-removed samples. ;

[0067] After outlier removal, each sequence type is processed using a length of [length missing]. The sliding window of each sampling point is normalized, and the right edge of the window is aligned with the current sampling point. The existing sampling points form a window. Within each window, the mean and standard deviation are calculated, and all sampling points within the window are standardized using the same mean and standard deviation. The standardization formula is as follows:

[0068] ;

[0069] in These are the normalized signal sample values. These are the signal sample values ​​after outlier removal. For The mean value within a sliding window consisting of sampling points. The standard deviation within the sliding window. Indoor combustible gas concentration signal sequence Terminal gas supply pressure signal sequence With gas flow signal sequence either of them, and when season To avoid division by zero and maintain numerical stability, the output consists of three types of signal sequences that are time-aligned and have undergone denoising, outlier removal, and normalization. .

[0070] In this specific embodiment, S2 includes:

[0071] The end-point gas supply pressure signal sequence is time-aligned and has undergone noise reduction, outlier removal, and normalization. With gas flow signal sequence Based on this, a neural state space model is constructed. The neural state space model uses state variables to represent the background component of the terminal gas supply pressure and uses a neural network to describe the relationship between state evolution and observation mapping.

[0072] The state variable is defined as follows:

[0073] ;

[0074] in For the first The state vector corresponding to each sampling time point For the transpose operator, This represents the background state of the terminal gas supply pressure and is used to characterize the slowly varying background component of the terminal gas supply pressure. It represents a changing state and is used to characterize the changing trend of the slowly varying background component;

[0075] The state transition function and observation function of the neural state-space model are parameterized using a neural network and satisfy the following:

[0076] ;

[0077] in It is a state transition function and is implemented by a state transition network. The set of network parameters for the state transition network. In order to be with the first Normalized sampled values ​​of gas flow rate at each sampling time. This is the predicted value of the terminal gas supply pressure and the prediction result used to characterize the background component of the terminal gas supply pressure. It is an observation function and is implemented by an observation network. The set of network parameters for the observation network;

[0078] The state transition network adopts a multilayer perceptron structure and uses... As input vector, As the output vector, the network layer and parameter values ​​are fixed at two hidden layers with 32 hidden units in each layer, and the activation function of the hidden layers is... and The output layer is a linear layer to ensure that the state variables take real values ​​and to adapt to the continuous state estimation of Kalman filtering;

[0079] The observation network adopts a multilayer perceptron structure and uses... As input vector, in scalar As output, the network layer and parameter values ​​are fixed at one hidden layer with 16 hidden units, the hidden layer activation function is ReLU, and the output layer is a linear layer. The dimensionless predicted end-supply pressure value will then be used in subsequent steps. Compared with observed values Establishing correlations for estimation of pressure background components;

[0080] The initial state estimation The value is determined by the statistics of the terminal gas supply pressure signal sequence in the initial time period, and the length of the initial time period is taken as... One sampling point, The first component is taken as exist to The arithmetic mean within the range is used to initialize the terminal gas supply pressure background state. The second component is taken as exist to The arithmetic mean within the range is used to initialize the changing state;

[0081] The initial error covariance is denoted as And set as A diagonal matrix, with the first diagonal element set to 0.25 to represent the diagonal matrix. The confidence level of the initial value is set to 0.01 for the second diagonal element to characterize the confidence level. The confidence level of the initial value is set to 0 for off-diagonal elements to indicate that the initial value errors of the two states are uncorrelated;

[0082] Furthermore, before model deployment, the state transition network and observation network were trained offline using data containing only normal gas usage conditions. The training data input was... and The training objective function adopts and The mean squared error is calculated and parameters are updated using the Adam optimizer, with the learning rate set to... Batch size is 256, training rounds are 50, and the results after training are obtained. and The neural Kalman filter recursive estimate is solidified for subsequent steps.

[0083] In this specific embodiment, S3 includes:

[0084] Based on the neural state-space model, the terminal gas supply pressure signal sequence Perform neural Kalman filtering recursive estimation to obtain the estimated sequence of the background component of the terminal gas supply pressure, recursively estimating the initial state. With initial error covariance As initial conditions;

[0085] For each sampling time The prediction update and correction update are performed sequentially, and the corresponding background component estimate is output after each update.

[0086] The neural Kalman filter employs a state transition function parameterized from the neural network. With observation function The extended Kalman filter is implemented by first-order linearization, wherein the Jacobian matrix required for first-order linearization is calculated using automatic differentiation and the network parameters trained and fixed offline in step S2 are fixed during inference. and ;

[0087] Using a covariance estimation network Output adaptive process noise covariance With adaptive observation noise covariance Covariance estimation network It is a multilayer perceptron with a two-dimensional vector as input. ,in The difference between the predicted and observed values ​​of the terminal gas supply pressure is used to characterize innovative items. For the first The normalized sampled values ​​of gas flow rate at each sampling time point are used. The covariance estimation network contains one hidden layer with 16 hidden units. The activation function of the hidden layer is ReLU. The output layer is a linear layer and outputs three scalars to determine the gas flow rate at each sampling time point. diagonal element and The value of is denoted as . Furthermore, it is solidified through offline training before deployment;

[0088] The recursive calculation process is shown in the following formula:

[0089] ,

[0090] ,

[0091] ,

[0092] ,

[0093] ,

[0094] ,

[0095] ;

[0096] in For the first The predicted state vector at each sampling time and the state variable in step S2. Consistent in both dimension and meaning For the first The correction state vector at each sampling time. The terminal gas supply pressure is the predicted value output by the observation function. The normalized observation value of the terminal gas supply pressure. The raw output of the covariance estimation network, softplus The activation function used to ensure that the covariance is positive. A lower bound constant to prevent the covariance from taking zero. and These represent the variances of the process noise covariance in the background state and the changing state dimensions, respectively. To observe the variance of the noise covariance, Construct operators for diagonal matrices. for The process noise covariance matrix, For scalar observation noise covariance, The first-order partial derivatives of the state transition function with respect to the state vector are formed. Jacobian matrix The first-order partial derivatives of the observation function with respect to the state vector constitute the... Jacobian matrix For the prediction error covariance matrix, To correct the error covariance matrix, Here is the Kalman gain matrix. for identity matrix To find the inverse operator and because Since it is a scalar, it is implemented by inverting the scalar;

[0097] At each sampling time The estimated sequence of background components of terminal gas supply pressure is obtained by arranging them in chronological order. .

[0098] In this specific embodiment, S4 includes:

[0099] Estimation sequence of background component of terminal gas supply pressure With end-supply pressure signal sequence Background stripping was performed to obtain pressure residual signal sequences, both of which were based on a unified time axis. Construct and correspond one-to-one at the sampling time, that is, for any sampling sequence number All observations exist at the same time. Compared with background estimates ;

[0100] Background stripping is achieved through point-by-point subtraction, and a pressure residual signal is output. The calculation method is as follows:

[0101] ;

[0102] in For the first The pressure residual signal at each sampling time is used to characterize the residual fluctuation component in the terminal gas supply pressure relative to the background component. For the first Normalized observations of the terminal gas supply pressure at each sampling time. For the first Estimated values ​​of background components of terminal gas supply pressure at each sampling time;

[0103] For all sampling numbers The pressure residual signal sequence was obtained by calculating and arranging the signals sequentially in chronological order. The pressure residual signal sequence and the terminal gas supply pressure signal sequence have the same set of sampling times.

[0104] In this specific embodiment, S5 includes:

[0105] Pressure residual signal sequence corresponding to each point under a unified time axis Indoor combustible gas concentration signal sequence With gas flow signal sequence Synchronous sliding segmentation and multimodal feature fusion are performed to form a multimodal fused feature sequence; synchronous sliding segmentation uses a fixed window length. Each sampling point and a fixed sliding step size There are 1 sampling points, among which This represents the number of sampling points covered by each sliding time window, and due to the steps... Medium resampling interval Therefore, the fixed time span corresponds to 120 seconds. This represents the sampling point interval between the starting positions of adjacent sliding time windows, corresponding to a sliding interval of 30 seconds. The window number is denoted as... And the The starting sampling number of each sliding time window is denoted as... If and only if The sliding time window is generated and the three types of signals are sampled within the specified interval. Internal synchronous values ​​are obtained to obtain one-to-one corresponding pressure residual signal subsequence, indoor combustible gas concentration signal subsequence and gas flow signal subsequence;

[0106] Within each sliding time window, statistical features characterizing temporal changes are extracted for each type of signal, and the features of the three types of signals are concatenated in a fixed order to achieve multimodal feature fusion. For any signal... Feature extraction is based on the signal within the current sliding time window. Each sample value, the signal In step S5, the value is taken as and One of the three, and the meanings of the three are pressure residual, normalized indoor combustible gas concentration and normalized gas flow rate, respectively;

[0107] The statistical features include the mean, standard deviation, maximum, minimum, and first-order difference statistics within the window. The first-order difference statistics are represented by the mean and standard deviation of the difference sequences of adjacent sampling points within the window. These difference sequences are obtained by subtracting adjacent sampling points within the window and are used to characterize the rate of change and fluctuation of the signal within the window. The six features corresponding to the pressure residual signal, the six features corresponding to the indoor combustible gas concentration signal, and the six features corresponding to the gas flow rate signal are concatenated in the order of "pressure residual first, concentration in the middle, flow rate last" to form the multimodal fusion feature vector of the sliding time window and constitute the multimodal fusion feature sequence. The multimodal fusion feature vector is denoted as:

[0108] ;

[0109] in For the first The multimodal fusion feature vectors of sliding time windows have a dimension of . For vector concatenation operators, This is a six-dimensional feature vector extracted from the pressure residual signal subsequence. This is a six-dimensional feature vector extracted from the indoor combustible gas concentration signal subsequence. This is a six-dimensional feature vector extracted from a subsequence of gas flow signals, and each... The six elements correspond to the mean, standard deviation, maximum value, minimum value, first difference mean, and first difference standard deviation in sequence.

[0110] All sliding time windows according to The multimodal fusion feature sequence is obtained by arranging the features in ascending time order. .

[0111] In this specific embodiment, S6 includes:

[0112] Multimodal fusion feature sequences Input the residual leakage discrimination network to output leakage score and discrimination result, and simultaneously output gas consumption behavior category result;

[0113] in For the first The multimodal fusion feature vectors corresponding to each sliding time window have a dimension of . This refers to the sequence number of the sliding time window;

[0114] The residual leakage discrimination network consists of a temporal feature extraction subnetwork and a discrimination subnetwork. The temporal feature extraction subnetwork uses a two-layer gated recurrent unit network. For temporal modeling, the hidden state dimension of the first-layer gated recurrent unit is set to 64, and its output sequence is used as the input of the second-layer gated recurrent unit. The hidden state dimension of the second-layer gated recurrent unit is set to 64, and its hidden state at the last time step is used as the temporal feature representation. ,in For the first Each time window corresponds to a temporal feature representation with a dimension of 64;

[0115] Network inference uses a fixed sequence length Each time window is used as an input segment and formed ,when The missing preceding time window is padded to the left by the zero vector to ensure that the input dimension remains constant.

[0116] The residual leak detection network sets up a shared temporal feature extraction subnetwork and connects two task heads at its output to achieve the joint output of the leak detection task and the gas consumption behavior recognition task. The gas consumption behavior recognition task head is implemented using a two-layer fully connected structure. The output dimension of the first fully connected layer is 32 and the activation function is ReLU. The output dimension of the second fully connected layer is 3, and the probability vector of the gas consumption behavior category is obtained through softmax. ,in and These represent the probabilities of three types of gas usage behavior: ignition gas usage, steady-state gas usage, and valve closure shutdown. The results of these gas usage behavior categories are as follows. Take as The category index corresponding to the highest probability in the middle;

[0117] The leak detection task header uses the gas behavior category result as auxiliary input when outputting the leak score. and The fusion vector is obtained by concatenating the vectors. Leakage score is output through a two-layer fully connected structure. The first fully connected layer has an output dimension of 32 and uses ReLU as the activation function. The second fully connected layer has an output dimension of 1, and a leakage score is obtained using sigmoid. The leakage score is calculated as follows:

[0118] ;

[0119] in For the first Leakage score for each time window, with a value range of [value missing]. It is the sigmoid function and is used to map real numbers to... interval, Represented by time series features Probability vector of gas usage behavior categories The concatenated fusion vector has a dimension of The weight vector of the second fully connected layer has a dimension of . For the transpose operator, This is the bias scalar for the second fully connected layer;

[0120] Leakage rating Compared with the preset classification threshold Compare and generate discrimination results ,in For the first The results of the time window are used to characterize normal gas usage or gas leak conditions. season And it was determined to be a gas leak condition, when season And it was determined to be a normal gas usage state;

[0121] The residual leakage discrimination network was trained offline using labeled samples and its parameters were fixed before deployment. The sample labels included leakage labels. Gas usage behavior labels ,in Indicates a gas leak status and This indicates normal gas usage. Indicates the use of gas for ignition and Indicates steady-state gas consumption and This indicates that the valve is shut off. The training objective is a weighted sum of the cross-entropy loss from the leakage task and the cross-entropy loss from the gas usage behavior task, with weights of 1 and 0.3 respectively. The optimizer is Adam, and the learning rate is [missing value]. The batch size is set to 128 and the number of training epochs is set to 40. After training, the network parameters are fixed and used for online inference to output leakage score sequences. , discrimination result sequence Sequence of results related to gas usage behavior categories .

[0122] In this specific embodiment, S7 includes:

[0123] Based on leakage scoring sequence With the discrimination result sequence The execution time is accumulated and a threshold judgment is made to output the gas leak warning result for the household;

[0124] in This refers to the sliding time window number. For the first Leakage score for each time window, with a value range of [value missing]. For the first The discrimination results of each time window and Indicates the gas leak status and This indicates normal gas usage.

[0125] The cumulative leakage score is generated using an exponentially weighted cumulative rule over time. and compare it with the preset decision threshold. The comparison is performed by counting the number of times the judgment result is a gas leak state within a preset judgment time window and comparing it with a preset number threshold.

[0126] The index-weighted cumulative calculation is based on the following recursive relationship:

[0127] ;

[0128] in For the first The cumulative leakage score corresponding to each time window, with a value range of [value missing]. The exponential weighting coefficients are taken as follows: This allows the cumulative leakage score to decay and remember historical scores. This is the cumulative leakage score for the previous time window. Score the leakage for the current time window, and set the initial conditions as follows: ;

[0129] The preset decision time window is defined by its length based on the time window sequence number. And at every moment Statistical interval Number of times gas leaks are detected inside ,in For the interval to satisfy The number of terms and when The missing pre-processor will be used. Treat it as 0 to ensure that the statistical rules are determined;

[0130] Preset number of times threshold If and only if satisfying and The system outputs the gas leak warning result for the household and marks it as a warning status, and records the warning flag as [missing information]. Otherwise, the gas leak warning result will be output as a non-warning state and the warning flag will be recorded as... ,in For the first The system outputs early warning results for each time window and provides them to the host computer or local linkage module for audible and visual alarms and shut-off valve control.

[0131] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

[0132] This invention addresses the false alarms and missed alarms caused by "normal gas usage disturbances superimposed on pressure background fluctuations" in in-home gas monitoring. It decouples the background modeling of the terminal gas supply pressure from leak detection: First, a neural state-space model is constructed based on the terminal gas supply pressure signal and gas flow signal. The pressure background component is recursively estimated using neural Kalman filtering, enabling the model to track slowly changing pressure backgrounds even when disturbances such as pipeline pressure regulation and neighboring household gas usage exist. Then, background stripping is performed on the pressure signal to obtain the pressure residual signal, enhancing the separability of abnormal fluctuations caused by leaks in the pressure domain. Based on this, the pressure residual, indoor combustible gas concentration signal, and gas flow signal are synchronously segmented according to time windows and feature fusion is performed. The residual is input into a residual leak detection network, which outputs a leak score and detection result. By combining the time accumulation of the leak score with threshold judgment, continuous evidence accumulation is achieved, thereby improving the accuracy of identifying real leaks in complex gas usage scenarios and providing a more stable early warning effect for minor leaks and slow seepage.

[0133] Compared to existing technologies that only use a single threshold or simple rules, this invention makes targeted improvements to the algorithm structure to address the aforementioned technical problems: First, it introduces gas flow rate as an exogenous input into the state transition process and sets state variables to characterize the pressure background and its changes, enabling the background estimation to be adaptively updated with changes in gas consumption, reducing the risk of missed detections caused by background drift; Second, it uses a neural network to parameterize the state transition function and observation function to characterize the nonlinear relationship between the pressure and flow rate at the inlet, improving the fitting ability of the background estimation under different users and different operating conditions; Third, it can further improve the robustness of filtering when noise levels change by outputting adaptive process noise covariance or observation noise covariance through the covariance estimation network; Fourth, it can introduce auxiliary outputs or constraints of gas consumption behavior categories into the residual leakage discrimination network, enabling the model to maintain stable discrimination of leakage evidence even when normal transient disturbances such as ignition and valve closure occur, thereby further reducing false alarms and improving the consistency of early warnings.

Claims

1. A method for detecting in-home gas leaks based on multimodal fusion and deep learning, characterized in that, include: S1. In the scenario of in-home gas monitoring, acquire the indoor combustible gas concentration signal sequence, the terminal gas supply pressure signal sequence, and the gas flow signal sequence, and preprocess them to achieve time alignment; S2. Construct a neural state-space model based on the terminal gas supply pressure signal sequence and the gas flow signal sequence. The neural state-space model includes state variables to characterize the background component of the terminal gas supply pressure, a state transition function parameterized by a neural network, and an observation function parameterized by a neural network; S3. Use the neural state-space model to perform neural Kalman filtering recursive estimation on the terminal gas supply pressure signal sequence to obtain the estimated sequence of the background component of the terminal gas supply pressure. S4. Based on the estimated sequence of the terminal gas supply pressure background component, the terminal gas supply pressure signal sequence is stripped of background to obtain the pressure residual signal sequence, which is the difference sequence between the terminal gas supply pressure signal sequence and the estimated sequence of the terminal gas supply pressure background component; S5. The pressure residual signal sequence, the indoor combustible gas concentration signal sequence, and the gas flow signal sequence are segmented according to a preset time window, and multimodal feature fusion is performed within the time window to obtain a multimodal fused feature sequence; S6. The multimodal fused feature sequence is input into the residual leakage discrimination network, and the leakage score and discrimination result are output. The discrimination result is used to characterize the normal gas use state or the gas leakage state; S7. The leakage score is accumulated over time and compared with a preset judgment threshold. Combined with the discrimination result, the in-home gas leakage early warning result is output.

2. The method for identifying in-home gas leaks based on multimodal fusion and deep learning according to claim 1, characterized in that, S1 includes: The sampling times for the indoor combustible gas concentration signal sequence, the terminal gas supply pressure signal sequence, and the gas flow signal sequence are obtained respectively, and the three are resampled based on a unified time axis. When any signal sequence has missing sampling points on the unified time axis, interpolation is used to fill in the missing sampling points to complete time alignment; After time alignment is completed, the indoor combustible gas concentration signal sequence, the terminal gas supply pressure signal sequence, and the gas flow signal sequence are low-pass filtered to remove high-frequency noise and complete the noise reduction process. After denoising, the median and median absolute deviation of each signal sequence are calculated within a sliding window of a preset length. Sampling points that deviate from the median and whose deviation exceeds a preset multiple of the median absolute deviation are identified as outliers, and the outliers are replaced with the median within the sliding window to complete the outlier removal. After outlier removal, the mean and standard deviation of each signal sequence are calculated within a sliding window of a preset length. The sampling points within the window are then processed by subtracting the mean and dividing by the standard deviation to complete the normalization process.

3. The method for identifying in-home gas leaks based on multimodal fusion and deep learning according to claim 1, characterized in that, S2 include: The state variables of the neural state space model include the end-supply pressure background state, which characterizes the end-supply pressure background component, and the change state, which characterizes the end-supply pressure background component over time. The state transition function is parameterized by a neural network and is used to update the state variable at the current sampling time based on the state variable at the previous sampling time and the gas flow rate signal sequence at the current sampling time. The observation function is parameterized by a neural network and is used to generate a predicted value of the terminal gas supply pressure based on the state variable at the current sampling time. The predicted value of the terminal gas supply pressure is associated with the observed value at the corresponding sampling time of the terminal gas supply pressure signal sequence, so that the neural state space model can be used to characterize the background component of the terminal gas supply pressure. The initial state estimation is determined based on the statistics of the terminal gas supply pressure signal sequence in the initial time period, and the initial error covariance is determined based on a preset covariance parameter, wherein the preset covariance parameter is used to characterize the confidence level of the initial state estimation.

4. The method for identifying in-home gas leaks based on multimodal fusion and deep learning according to claim 1, characterized in that, S3 include: The neural Kalman filter recursive estimation uses the initial state estimation and the initial error covariance as initial conditions, and performs prediction update and correction update sequentially for each sampling time of the terminal gas supply pressure signal sequence; In the prediction update, based on the state transition function of the neural state space model and combined with the value of the gas flow signal sequence at the current sampling time, the state variable at the previous sampling time is predicted to obtain the predicted state variable at the current sampling time, and the prediction error covariance is updated based on the predicted state variable and the process noise covariance. In the correction update, the predicted state variables are mapped to predicted values ​​of end-gas pressure based on the observation function of the neural state space model. Based on the difference between the predicted value of the terminal gas supply pressure and the observed value of the terminal gas supply pressure signal sequence at the corresponding sampling time, the Kalman gain is calculated by combining the prediction error covariance and the observation noise covariance, and the predicted state variable is corrected by the Kalman gain to obtain the corrected state variable, and the correction error covariance is updated at the same time. Based on the corrected state variables at each sampling time, the estimated sequence of the background component of the terminal gas supply pressure is output.

5. The method for identifying in-home gas leaks based on multimodal fusion and deep learning according to claim 1, characterized in that, S4 include: The estimated sequence of the background component of the terminal gas supply pressure corresponds one-to-one with the signal sequence of the terminal gas supply pressure at the same sampling time. For any sampling time, the value of the terminal gas supply pressure signal sequence at that sampling time is subtracted from the value of the terminal gas supply pressure background component estimation sequence at that sampling time to obtain the pressure residual signal corresponding to that sampling time. Thus, the pressure residual signal sequence is composed of the pressure residual signals at each sampling time, wherein the pressure residual signal sequence and the terminal gas supply pressure signal sequence have the same set of sampling times.

6. The method for identifying in-home gas leaks based on multimodal fusion and deep learning according to claim 1, characterized in that, S5 include: The pressure residual signal sequence, indoor combustible gas concentration signal sequence, and gas flow signal sequence are synchronously slid segmented with a preset window length and a preset sliding step size, so that within any sliding time window, the pressure residual signal subsequence, indoor combustible gas concentration signal subsequence, and gas flow signal subsequence corresponding to that sliding time window are obtained respectively, and the pressure residual signal subsequence, the indoor combustible gas concentration signal subsequence, and the gas flow signal subsequence correspond one-to-one at the sampling time; Within each sliding time window, features for characterizing temporal changes are extracted from the pressure residual signal subsequence, the indoor combustible gas concentration signal subsequence, and the gas flow rate signal subsequence. These features include mean, standard deviation, maximum value, minimum value, and first-order difference statistics. The features corresponding to the pressure residual signal subsequence, the indoor combustible gas concentration signal subsequence, and the gas flow rate signal subsequence are then concatenated in a preset order to obtain the multimodal fusion features corresponding to that sliding time window. The multimodal fusion features corresponding to each sliding time window are arranged in chronological order to obtain the multimodal fusion feature sequence.

7. The method for identifying in-home gas leaks based on multimodal fusion and deep learning according to claim 1, characterized in that, S6 include: The residual leakage discrimination network includes a temporal feature extraction subnetwork and a discrimination subnetwork. The temporal feature extraction subnetwork is used to perform temporal modeling on the multimodal fusion feature sequence and output a temporal feature representation. The discrimination subnetwork is used to classify the temporal feature representation and output a leakage score. The leak score is a probability value or confidence value that characterizes the state of a gas leak. A judgment result is generated based on the comparison result between the leakage score and the preset classification threshold. When the leakage score is not less than the preset classification threshold, the judgment result is determined to be a gas leakage state. When the leakage score is less than the preset classification threshold, the judgment result is determined to be a normal gas usage state.

8. The method for identifying in-home gas leaks based on multimodal fusion and deep learning according to claim 1, characterized in that, S7 includes: The leakage scores are arranged in chronological order to form a leakage score sequence, and the leakage score sequence is accumulated over time based on a preset accumulation rule to obtain a cumulative leakage score. The preset accumulation rule includes at least one of sliding window accumulation and exponential weighted accumulation. The cumulative leakage score is compared with a preset judgment threshold, and the number of times the judgment result is a gas leakage state within the preset judgment time window is counted. When the cumulative leakage score is not less than the preset judgment threshold, and the number of times the judgment result is gas leakage state within the preset judgment time window is not less than the preset number threshold, the gas leakage warning result for entering the house is output as a warning state. When the cumulative leakage score is less than the preset judgment threshold, or when the number of times the judgment result is a gas leakage state within the preset judgment time window is less than the preset number threshold, the gas leakage warning result for entering the house is output as a non-warning state.

9. A method for identifying in-home gas leaks based on multimodal fusion and deep learning according to claim 4, characterized in that, At least one of the process noise covariance and the observation noise covariance is an adaptive covariance, which is output by a covariance estimation network. The input to the covariance estimation network includes the difference between the predicted value of the terminal gas supply pressure and the observed value of the terminal gas supply pressure signal sequence, and at least one of the gas flow signal sequence.

10. The method for identifying in-home gas leaks based on multimodal fusion and deep learning according to claim 7, characterized in that, The residual leakage discrimination network also outputs gas consumption behavior category results, which include at least one of ignition gas consumption, steady-state gas consumption, and valve shutdown. Furthermore, when outputting a leakage score, the discriminative subnetwork uses the gas usage behavior category result as an auxiliary input or auxiliary task constraint to reduce false alarms caused by normal gas usage disturbances.

Images