Hydrogen-mixed natural gas pipeline leakage detection method and system based on multi-task mamba-cnn

Abstract

The present application relates to the field of gas detection, and discloses a kind of multi-task Mamba-CNN's hydrogen mixed natural gas pipeline leak detection method and system.The method first collects the audio data corresponding to pipeline, and converts the audio data into Log-Mel spectrogram (logarithmic mel frequency spectrum), then processes Log-Mel spectrogram by leak detection model to determine whether pipeline leaks, wherein, leak detection model is based on first layer normalization layer, linear transformation layer, state space sequence module, convolutional neural network (CNN) and multi-task classification head is constructed, can capture long-range temporal dependencies by time series modeling, improve the identification ability of hydrogen mixed natural gas pipeline leak characteristics.

Description

Technical Field

[0001] This invention relates to the field of gas detection, and relates to, but is not limited to, a method and system for detecting leaks in hydrogen-blended natural gas pipelines based on multi-task Mamba-CNN. Background Technology

[0002] Hydrogen transportation, a crucial component of hydrogen utilization, is also a key factor limiting the development of hydrogen energy. Common methods of hydrogen transportation include gaseous and liquid forms: high-pressure cylinders are simple to manufacture and particularly suitable for short-distance, small-volume supply. However, liquid hydrogen transportation is costly and unsuitable for large-scale distribution. Co-transporting hydrogen in a certain proportion into existing natural gas pipeline networks is an effective method for long-distance hydrogen transport, reducing not only the construction cost of dedicated hydrogen pipelines but also carbon emissions from natural gas. Due to the unique properties of hydrogen, mixing it into natural gas pipelines can cause hydrogen embrittlement, threatening the structural integrity of the pipelines. Furthermore, hydrogen can easily diffuse through tiny cracks, leading to safety issues and even leaks in the pipeline. Summary of the Invention

[0003] In view of this, embodiments of the present invention provide a multi-task Mamba-CNN method for detecting leaks in hydrogen-blended natural gas pipelines, which can detect leaks in hydrogen-blended natural gas pipelines and improve pipeline safety.

[0004] The specific technical solutions of this invention are as follows:

[0005] The first aspect of this application provides for obtaining audio data corresponding to the pipeline;

[0006] The audio data is processed to obtain the corresponding log-Mel spectrum.

[0007] The log-Mel spectrum is processed based on the leak detection model to obtain the detection results; wherein, the leak detection model is constructed based on a first normalization layer, a linear transformation layer, a state space sequence module, a convolutional neural network, and a multi-task classification head, and the detection results include leak detection results, hydrogen mixing ratio classification results, and pressure level classification results.

[0008] In some embodiments, processing the log-Melogram based on the leakage detection model to obtain the detection result includes:

[0009] The log-Mel spectrum is normalized by the first normalization layer to obtain the normalized log-Mel spectrum.

[0010] The normalized log-Mel spectrum is projected into a low-dimensional latent space through the linear transformation layer to obtain the low-dimensional feature sequence corresponding to the normalized log-Mel spectrum.

[0011] The low-dimensional feature sequence is processed by the state space sequence module to obtain the enhanced feature sequence corresponding to the low-dimensional feature sequence;

[0012] The enhanced feature sequence is processed by the convolutional neural network to perform feature extraction and adaptive average pooling operations to obtain the high-level feature vector corresponding to the enhanced feature sequence.

[0013] The high-level feature vector is classified using the multi-task classification head to obtain the detection result.

[0014] In some embodiments, the state space sequence module includes a first state space sequence unit and a second state space sequence unit; the second state space sequence unit is stacked in series with the first state space sequence unit.

[0015] The step of processing the low-dimensional feature sequence through the state space sequence module to obtain the enhanced feature sequence corresponding to the low-dimensional feature sequence includes:

[0016] The low-dimensional feature sequence is temporally modeled and normalized using the first state space sequence unit to obtain the initial enhanced feature sequence corresponding to the low-dimensional feature sequence.

[0017] The initial enhanced feature sequence is further subjected to temporal modeling and normalization processing by the second state space sequence unit to obtain the enhanced feature sequence.

[0018] In some embodiments, the first state space sequence unit includes a sequence modeling layer and a second normalization layer;

[0019] The step of performing time-series modeling and normalization on the low-dimensional feature sequence using the first state-space sequence unit to obtain the first feature sequence corresponding to the low-dimensional feature sequence includes:

[0020] The sequence modeling layer is used to perform time-series modeling on the low-dimensional feature sequence to obtain the initial feature sequence.

[0021] The initial feature sequence is normalized by the second normalization layer to obtain the first feature sequence.

[0022] In some embodiments, the convolutional neural network includes a plurality of convolutional blocks and an adaptive average pooling layer;

[0023] The step of performing feature extraction and adaptive average pooling operations on the enhanced feature sequence through the convolutional neural network to obtain the high-level feature vector corresponding to the enhanced feature sequence includes:

[0024] The enhanced feature sequence is processed by multiple convolutional blocks to obtain an initial high-level feature vector.

[0025] The high-level feature vector is obtained by performing adaptive average pooling on the initial high-level feature vector through the adaptive average pooling layer.

[0026] In some embodiments, the plurality of convolutional blocks include a first convolutional block, a second convolutional block, and a third convolutional block, wherein the number of channels corresponding to the first convolutional block is less than the number of channels corresponding to the second convolutional block, and the number of channels corresponding to the second convolutional block is less than the number of channels corresponding to the third convolutional block.

[0027] The step of performing feature extraction processing on the enhanced feature sequence through multiple convolutional blocks to obtain an initial high-level feature vector includes:

[0028] The enhanced feature sequence is processed by the first convolutional block to obtain the first-level features;

[0029] The second convolutional block is used to extract features from the first-level features to obtain the second-level features.

[0030] The second-level features are extracted using the third convolutional block to obtain the initial high-level feature vector.

[0031] In some embodiments, the first convolutional block includes a depth-separable convolutional layer, a batch normalization layer, an activation layer, a pooling layer, and a dropout layer;

[0032] The step of performing feature extraction processing on the enhanced feature sequence through the first convolutional block to obtain first-level features includes:

[0033] The enhanced feature sequence is extracted by the depthwise separable convolutional layer to obtain the first feature sequence.

[0034] The first feature sequence is batch normalized through the batch normalization layer to obtain the second feature sequence;

[0035] The second feature sequence is activated by the activation layer to obtain the third feature sequence;

[0036] The third feature sequence is pooled using the pooling layer to obtain the fourth feature sequence;

[0037] The fourth feature sequence is regularized using the discard layer to obtain the first-level feature.

[0038] In some embodiments, the multi-task classification head includes a first classification head, a second classification head, and a third classification head;

[0039] The process of classifying the high-level feature vector using the multi-task classification head to obtain detection results includes:

[0040] The leak detection result is obtained by classifying the high-rise feature vector using the first classification head;

[0041] The hydrogen mixing ratio is classified by the second classification head on the high-level feature vector to obtain the hydrogen mixing ratio classification result;

[0042] The pressure level classification result is obtained by classifying the high-level feature vector using the third classification head.

[0043] In some embodiments, processing the audio data to obtain the log-Melbourne spectrogram corresponding to the audio data includes:

[0044] The audio data is segmented into frames, and a window function is applied to each frame to obtain multiple frame data corresponding to the audio data.

[0045] Perform short-time Fourier transform on each of the multiple frame data to obtain multiple spectra corresponding to the multiple frame data;

[0046] Multiple power spectra corresponding to multiple spectra are determined, and the multiple power spectra are nonlinearly mapped and weighted summed using a Mel filter to obtain a Mel spectrum.

[0047] The logarithmic Mel spectrogram is obtained by logarithmically scaling the energy value of each frequency band in the Mel spectrogram.

[0048] A second aspect of this application provides a hydrogen-blended natural gas pipeline leak detection system based on multi-task Mamba-CNN, the system comprising:

[0049] The acquisition module is used to acquire the audio data corresponding to the pipe.

[0050] The processing module is used to process the audio data to obtain the log-Mel spectrum corresponding to the audio data;

[0051] The detection module is used to process the log-Mel spectrum based on the leak detection model to obtain the detection result; wherein, the leak detection model is constructed based on a first normalization layer, a linear transformation layer, a state space sequence module, a convolutional neural network, and a multi-task classification head, and the detection result includes leak detection result, hydrogen mixing ratio classification result, and pressure level classification result.

[0052] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following:

[0053] In this embodiment of the invention, audio data corresponding to the pipeline is first collected and converted into a Log-Mel spectrum. Then, the Log-Mel spectrum is processed by a leak detection model to determine whether the pipeline is leaking. The leak detection model is constructed based on a first normalization layer, a linear transformation layer, a state space sequence module, a convolutional neural network, and a multi-task classification head. It can capture remote time dependencies through time series modeling and improve the ability to identify leak characteristics of hydrogen-blended natural gas pipelines. Attached Figure Description

[0054] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort, wherein:

[0055] Figure 1 A flowchart illustrating the multi-task Mamba-CNN method for detecting leaks in hydrogen-doped natural gas pipelines provided in this embodiment of the invention;

[0056] Figure 2 This is a schematic diagram of the structure of a leakage detection model provided in an embodiment of the present invention;

[0057] Figure 3 This is a schematic diagram of the structure of a sequence modeling layer provided in an embodiment of the present invention;

[0058] Figure 4 This is a schematic diagram of the structure of a hydrogen-blended natural gas pipeline leak detection system based on multi-task Mamba-CNN provided in an embodiment of the present invention;

[0059] Figure 5 This is a schematic diagram of a control device provided in an embodiment of the present invention. Detailed Implementation

[0060] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. The following embodiments are used to illustrate the present invention, but are not intended to limit the scope of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0061] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0062] It should be noted that the terms "first, second, and third" used in the embodiments of the present invention are only used to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, and third" can be interchanged in a specific order or sequence where permitted, so that the embodiments of the present invention described herein can be implemented in an order other than that illustrated or described herein.

[0063] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which these embodiments of the invention pertain. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art and should not be interpreted in an idealized or overly formal sense unless specifically defined as herein.

[0064] Mamba is a novel sequence model based on a selective state-space model. Its main goal is to reduce time and memory complexity while retaining Transformer-level expressiveness.

[0065] Log-Mel spectrograms are based on the short-time Fourier transform (STFT) and integrate the psychoacoustic characteristics of the human ear's greater sensitivity to low frequencies.

[0066] Multi-Task Learning (MTL) aims to jointly train a model on multiple related tasks, enabling it to leverage the inherent relationships and shared information between these tasks, thereby improving generalization and reducing overfitting. Compared to traditional single-task learning, MTL introduces cross-task information exchange and constraints at the model level, effectively treating each task as an additional supervisory signal on the same underlying representation.

[0067] Figure 1 This is a flowchart illustrating a multi-task Mamba-CNN-based method for detecting leaks in hydrogen-blended natural gas pipelines, as provided in an embodiment of the present invention. The method can be executed via a control device. The control device includes a multi-task Mamba-CNN-based hydrogen-blended natural gas pipeline leak detection system.

[0068] In some embodiments, the control device includes at least one of a personal computer, a laptop computer, a smartphone, a tablet computer, and a portable wearable device, but this application embodiment does not limit this.

[0069] like Figure 1 As shown, the multi-task Mamba-CNN method for detecting leaks in hydrogen-blended natural gas pipelines provided in this embodiment of the invention may include steps S101-S103.

[0070] S101. Obtain the audio data corresponding to the pipe.

[0071] In some embodiments, the control device can acquire audio data corresponding to the pipeline through a leak detection module. The leak detection module may include an acoustic sensor and a sound acquisition device; the acoustic sensor is used to collect sound signals from the environment surrounding the pipeline, and the sound acquisition device is coupled to the acoustic sensor to convert the sound signals collected by the acoustic sensor into digital signals to obtain audio data.

[0072] For example, the leak detection module can collect audio data from the environment surrounding the pipeline in real time according to a preset sampling frequency, and send the audio data to the control device. The control device obtains the audio data corresponding to the pipeline by receiving the audio data sent by the leak detection module. The sampling frequency can be 48 kHz. This application embodiment does not limit this.

[0073] S102. Process the audio data to obtain the corresponding log-Mel spectrum.

[0074] In some embodiments, after the control device acquires the audio data, it processes the audio data to obtain the corresponding log-Mel spectrum.

[0075] For example, after the control device acquires the audio data, it can segment the audio data and process it to obtain the corresponding log-Mel spectrum.

[0076] In some embodiments, processing audio data to obtain a log-Mel spectrum corresponding to the audio data includes: performing frame-by-frame processing on the audio data and applying a window function to each frame to obtain multiple frame data corresponding to the audio data; performing short-time Fourier transform on the multiple frame data respectively to obtain multiple spectra corresponding to the multiple frame data; determining multiple power spectra corresponding to the multiple spectra, and performing nonlinear mapping and weighted summation on the multiple power spectra through a Mel filter to obtain a Mel spectrum; and performing logarithmic scaling on the energy value of each frequency band in the Mel spectrum to obtain a log-Mel spectrum.

[0077] For example, the control device segments the audio data into frames based on a preset frame length (e.g., 50ms), dividing the audio data into multiple frames of the same length and applying a window function to each frame to obtain multiple frame data corresponding to the audio data. The window function is existing technology and will not be elaborated here. After the control device completes the framing and windowing of the audio data, it can perform a short-time Fourier transform on each frame data to obtain the spectrum corresponding to each frame data. When all frames data have undergone short-time Fourier transforms, multiple spectra corresponding to the multiple frames data will be obtained. After obtaining multiple spectra, the control device can calculate the power spectrum corresponding to each spectrum, thereby determining the multiple power spectra corresponding to the multiple spectra. It then uses a Mel filter to perform nonlinear mapping and weighted summation on the multiple power spectra to obtain a Mel spectrogram. After obtaining the Mel spectrogram, the control device can logarithmically scale the energy values ​​of each frequency band in the Mel spectrogram to obtain a logarithmic Mel spectrogram.

[0078] In some embodiments, the control device can calculate the power spectrum corresponding to each spectrum using Formula 1, as shown below:

[0079] (Formula 1);

[0080] in, This represents a time frame, and k represents the frequency. Indicates time frame The power spectrum at frequency k. Indicates time frame The complex STFT coefficients corresponding to the lower frequency k.

[0081] The control device can perform a weighted summation of multiple power spectra using Equation 2, as shown below:

[0082] (Formula 2);

[0083] in, This represents the time frame, k represents the frequency, and m represents the Mel filter number. This represents the weight of the Mel filter m at frequency k. Indicates time frame The power spectrum at frequency k. Indicates time frame The Mel spectrum corresponding to the lower Mel filter m, where N represents the length of the STFT.

[0084] The control device can obtain the log-Mel spectrum value using Formula 3, as shown below:

[0085] (Formula 3);

[0086] in, This represents a time frame, and m represents the number of the Mel filter. Indicates time frame Mel filter m corresponds to the logarithmic Mel spectrum. Indicates time frame The Mel spectrum corresponding to the lower Mel filter m, where ln represents the natural logarithm function, used to convert the Mel spectrum into a logarithmic value; This is a preset constant.

[0087] It is understood that converting audio data into a log-Mel spectrogram in this embodiment of the application can preserve the acoustic details in the audio data to the greatest extent and reduce the computational complexity of subsequent processing.

[0088] S103. The log-Mel spectrum is processed based on the leakage detection model to obtain the detection results.

[0089] In some embodiments, the leak detection model is constructed based on a first normalization layer, a linear transformation layer, a state-space sequence module, a convolutional neural network, and a multi-task classification head.

[0090] For example, Figure 2 A schematic diagram of a leak detection model is shown, such as... Figure 2 As shown, the leak detection model comprises a Layer Norm (i.e., the first normalization layer), a Linear Transformation Layer, at least two Mamba Blocks (i.e., state-space sequence modules) stacked in series, a CNN (i.e., convolutional neural network), and Multi-task Classification Heads. The Layer Norm normalizes the input log-Melogram spectrum to obtain a normalized log-Melogram spectrum. The Linear Transformation Layer projects the normalized log-Melogram spectrum into a low-dimensional latent space to obtain the corresponding low-dimensional feature sequence. The at least two stacked Mamba Blocks process the low-dimensional feature sequence to obtain the corresponding enhanced feature sequence. The CNN performs feature extraction and adaptive average pooling on the enhanced feature sequence to obtain the corresponding high-level feature vector. The Multi-task Classification Heads execute a predefined classification task, i.e., obtain the detection result based on the high-level feature vector.

[0091] In some embodiments, the log-Mel spectrum is processed based on the leakage detection model to obtain the detection result, including: normalizing the log-Mel spectrum through a first normalization layer to obtain a normalized log-Mel spectrum; projecting the normalized log-Mel spectrum into a low-dimensional latent space through a linear transformation layer to obtain a low-dimensional feature sequence corresponding to the normalized log-Mel spectrum; processing the low-dimensional feature sequence through at least two state space sequence modules to obtain an enhanced feature sequence corresponding to the low-dimensional feature sequence; performing feature extraction and adaptive average pooling operations on the enhanced feature sequence through a convolutional neural network to obtain a high-level feature vector corresponding to the enhanced feature sequence; and classifying the high-level feature vector through a multi-task classification head to obtain the detection result.

[0092] For example, after the control device executes S102 to obtain the log-Mel spectrum corresponding to the audio data, it can input the log-Mel spectrum into the leakage detection model. Upon receiving the log-Mel spectrum, the first normalization layer in the leakage detection model normalizes the log-Mel spectrum in the frequency band dimension to eliminate differences in data distribution between different frequency bands, resulting in a normalized log-Mel spectrum. Then, the linear transformation layer in the leakage detection model projects the normalized log-Mel spectrum into a low-dimensional latent space to reduce the dimensionality of the normalized log-Mel spectrum, thus obtaining the low-dimensional feature sequence corresponding to the normalized log-Mel spectrum. Next, at least two state-space sequence modules in the leak detection model process the low-dimensional feature sequence, that is, capture the long-range dependencies with linear complexity in the low-dimensional feature sequence to obtain the enhanced feature sequence. After obtaining the enhanced feature sequence, the convolutional neural network in the leak detection model performs feature extraction processing and adaptive average pooling operation on the enhanced feature sequence to obtain the high-level feature vector corresponding to the enhanced feature sequence. Finally, the multi-task classification head in the leak detection model performs the corresponding classification task based on the high-level feature vector to obtain the detection result.

[0093] In some embodiments, the state space sequence module includes a first state space sequence unit and a second state space sequence unit; the second state space sequence unit is stacked in series with the first state space sequence unit; the low-dimensional feature sequence is processed by at least two state space sequence modules to obtain an enhanced feature sequence corresponding to the low-dimensional feature sequence, including: performing time-series modeling and normalization processing on the low-dimensional feature sequence by the first state space sequence unit to obtain an initial enhanced feature sequence corresponding to the low-dimensional feature sequence; and performing time-series modeling and normalization processing on the initial enhanced feature sequence again by the second state space sequence unit to obtain the enhanced feature sequence.

[0094] For example, the state-space sequence module includes a first state-space sequence unit and a second state-space sequence unit. The first and second state-space sequence units are stacked in series. In practical applications, the linear transformation layer outputs a low-dimensional feature sequence to the first state-space sequence unit. After receiving the low-dimensional feature sequence, the first state-space sequence unit performs temporal modeling and normalization on the low-dimensional feature sequence to obtain an initial enhanced feature sequence, which is then input to the second state-space sequence unit. After receiving the initial enhanced feature sequence, the second state-space sequence unit performs temporal modeling and normalization on the initial enhanced feature sequence again to obtain the enhanced feature sequence.

[0095] It should be noted that the number of state space sequence units included in the state space sequence module can be set according to actual needs. This application embodiment does not limit the number of state space sequence units included in the state space sequence module.

[0096] In some embodiments, the first state-space sequence unit includes a sequence modeling layer and a second normalization layer. The first state-space sequence unit performs temporal modeling and normalization on the low-dimensional feature sequence to obtain an initial enhanced feature sequence, including: performing temporal modeling on the low-dimensional feature sequence through the sequence modeling layer to obtain an initial feature sequence; and performing normalization on the initial feature sequence through the second normalization layer to obtain an initial enhanced feature sequence.

[0097] For example, such as Figure 2 As shown, the first state-space sequence unit may include Mamba (equivalent to the sequence modeling layer mentioned above) and LN (Layer Norm, equivalent to the second normalization layer mentioned above). Mamba is used to perform temporal modeling on the low-dimensional feature sequence, thereby capturing the long-range dependencies corresponding to the low-dimensional feature sequence and obtaining the initial feature sequence. LN is used to normalize the initial feature sequence to obtain the initial enhanced feature sequence. After obtaining the initial enhanced feature sequence, LN outputs the initial enhanced feature sequence to Mamba in the second state-space sequence unit, so that the enhanced feature sequence can be obtained through the second state-space sequence unit.

[0098] It should be noted that the second state space sequence unit has a similar structure and function to the first state space sequence unit, which will not be elaborated here.

[0099] In some embodiments, Figure 3 A schematic diagram of a sequence modeling layer is shown, such as Figure 3As shown, the sequence modeling layer comprises two parallel branches: the main path and the residual path. The main path consists of a projection layer, a convolutional layer, a Silu (Sigmoid-Weighted Linear Unit) activation layer, and a State Space Model (SSM) connected in series. Following the SSM is a projection layer that transforms the features output by the SSM to the target space, serving as the output of the main path. The residual path consists of a projection layer and a Silu activation layer connected in series. In the residual path, the input data passes through the projection and Silu activation layers and is then multiplied element-wise with the output of the SSM in the main path. The multiplied data is then projected back to the original input dimension through the projection layer following the SSM, thus fusing with the output of the main path.

[0100] In some embodiments, the convolutional neural network includes multiple convolutional blocks and an adaptive average pooling layer; the convolutional neural network performs feature extraction processing and adaptive average pooling operation on the enhanced feature sequence to obtain a high-level feature vector corresponding to the enhanced feature sequence, including: performing feature extraction processing on the enhanced feature sequence through multiple convolutional blocks to obtain an initial high-level feature vector; and performing adaptive average pooling operation on the initial high-level feature vector through the adaptive average pooling layer to obtain a high-level feature vector.

[0101] In some embodiments, the plurality of convolutional blocks include a first convolutional block, a second convolutional block, and a third convolutional block, wherein the number of channels corresponding to the first convolutional block is less than the number of channels corresponding to the second convolutional block, and the number of channels corresponding to the second convolutional block is less than the number of channels corresponding to the third convolutional block.

[0102] For example, the number of channels corresponding to the first convolutional block can be 32, the number of channels corresponding to the second convolutional block can be 64, and the number of channels corresponding to the third convolutional block can be 128. It should be noted that the embodiments of this application do not limit this.

[0103] In some embodiments, feature extraction processing is performed on the enhanced feature sequence through multiple convolutional blocks to obtain an initial high-level feature vector, including: performing feature extraction processing on the enhanced feature sequence through a first convolutional block to obtain a first-level feature; performing feature extraction processing on the first-level feature through a second convolutional block to obtain a second-level feature; and performing feature extraction processing on the second-level feature through a third convolutional block to obtain an initial high-level feature vector.

[0104] For example, such as Figure 2As shown, the number of convolutional blocks can be three. The three convolutional blocks are DWConv-32+Pool (first convolutional block), DWConv-64+Pool (second convolutional block), and DWConv-128+Pool (third convolutional block), with DWConv-128+Pool concatenated with an AdaptiveAvg Pool. Specifically, DWConv-32+Pool has 32 channels, DWConv-64+Pool has 64 channels, and DWConv-128+Pool has 128 channels. In practical applications, the second state space sequence unit outputs an enhanced feature sequence to the CNN. After receiving the enhanced feature sequence, the CNN uses DWConv-32+Pool to convolve the enhanced feature sequence through 32 channels and then uses a pooling layer to pool the convolved enhanced feature sequence, thereby completing the feature extraction of the enhanced feature sequence and obtaining a 32-dimensional feature vector (i.e., the first-level feature). Then, DWConv-64+Pool uses 64 channels to convolve the 32-dimensional feature vector and then uses a pooling layer to pool the convolved 32-dimensional feature vector, thereby completing the feature extraction of the 32-dimensional feature vector and obtaining a 64-dimensional feature vector (i.e., the second-level feature). Finally, DWConv-128+Pool uses 128 channels to convolve the 64-dimensional feature vector and then uses a pooling layer to pool the convolved 64-dimensional feature vector, thereby completing the feature extraction of the 64-dimensional feature vector and obtaining a 128-dimensional feature vector (i.e., the initial high-level feature vector). Then, adaptive average pooling is performed on the initial high-level feature vectors to obtain high-level feature vectors.

[0105] It should be noted that the number of multiple convolutional blocks can be set according to actual needs, and the embodiments of this application do not limit the number of multiple convolutional blocks.

[0106] In some embodiments, the first convolutional block includes a depthwise separable convolutional layer, a batch normalization layer, an activation layer, a pooling layer, and a dropout layer. The first convolutional block is used to extract features from the enhanced feature sequence to obtain first-level features, including: extracting features from the enhanced feature sequence using the depthwise separable convolutional layer to obtain a first feature sequence; batch normalizing the first feature sequence using the batch normalization layer to obtain a second feature sequence; activating the second feature sequence using the activation layer to obtain a third feature sequence; pooling the third feature sequence using the pooling layer to obtain a fourth feature sequence; and regularizing the fourth feature sequence using the dropout layer to obtain the first-level features.

[0107] For example, the first convolutional block includes DWConv-32 (a 32-channel depthwise separable convolutional layer), a batch normalization layer, an activation layer, a pooling layer, and a dropout layer. These layers are connected in series. After receiving the enhanced feature sequence, the first convolutional block performs convolution on the enhanced feature sequence through its 32 channels using DWConv-32 to obtain the first feature sequence. The batch normalization layer then performs batch normalization on the first feature sequence to obtain the second feature sequence. The activation layer then performs non-linear activation on the second feature sequence using a Rectified Linear Unit (ReLU) function to obtain the third feature sequence. The pooling layer then pools the third feature sequence to extract key features and reduce the data volume, resulting in the fourth feature sequence. Finally, the dropout layer regularizes the fourth feature sequence, randomly discarding some features to prevent overfitting, thus obtaining the first-level features.

[0108] It should be noted that the structure of depth-separable convolutional layers is existing technology and will not be described in detail here. The second and third convolutional blocks have similar structures and functions to the first convolutional block and will not be described in detail here either.

[0109] It is understood that the leak detection model in this embodiment performs temporal modeling using two state space sequence units (the first state space sequence unit and the second state space sequence unit) included in the state space sequence module to capture long-range temporal dependencies. Furthermore, it processes the data at multiple scales using three depthwise separable convolutional layers (the first convolutional block, the second convolutional block, and the third convolutional block) to comprehensively extract both local and global features. This effectively improves the ability to identify leak characteristics in hydrogen-blended natural gas pipelines.

[0110] In some embodiments, the detection results include leak detection results, hydrogen mixture ratio classification results, and pressure level classification results. At least one multi-task classification head includes a first multi-task classification head, a second multi-task classification head, and a third multi-task classification head. The detection results are obtained by classifying the high-level feature vectors using at least one multi-task classification head, including: classifying the high-level feature vectors for natural gas leaks using the first multi-task classification head; classifying the high-level feature vectors for hydrogen mixture ratios using the second multi-task classification head; and classifying the high-level feature vectors for pressure levels using the third multi-task classification head.

[0111] For example, such as Figure 2As shown, the number of at least one multi-task classification heads can be three. The three multi-task classification heads are Leak (equivalent to the first multi-task classification head mentioned above), Mix (equivalent to the second multi-task classification head mentioned above), and Pressure (equivalent to the third multi-task classification head mentioned above). The first multi-task classification head, upon receiving the high-level feature vector, performs natural gas leak classification on the high-level feature vector to determine whether a natural gas leak has occurred, thus obtaining a leak detection result. The leak detection result includes Leak and No leak. The second multi-task classification head, upon receiving the high-level feature vector, performs hydrogen mixing ratio classification on the high-level feature vector to determine which gas is leaking, thus obtaining a hydrogen mixing ratio classification result. The hydrogen mixing ratio classification result includes natural gas (Compressed Natural Gas, CNG), 5% hydrogen-compressed natural gas (HCNG), 10% HCNG, and no leak. Here, 5% HCNG means that hydrogen accounts for 5% of the total HCNG volume, and 10% HCNG means that hydrogen accounts for 10% of the total HCNG volume. After receiving the high-level feature vector, the third multi-task classification head will classify the high-level feature vector according to its pressure level to determine the pressure level of the leak, thus obtaining the pressure level classification result. The pressure level classification results include 0.2 MPa, 0.1 MPa, and no leak.

[0112] It should be noted that the number of at least one multi-task classification heads can be set according to actual needs, and the classification tasks corresponding to each classification head can also be set according to actual needs. In this application embodiment, the number of at least one multi-task classification heads and the classification tasks corresponding to each classification head are not limited.

[0113] It is understood that the embodiments of this application, through multi-task learning, can simultaneously handle three different tasks: leak detection, hydrogen mixture ratio classification, and pressure level classification, thereby improving the efficiency of the leak detection model; and multi-task learning can utilize the inherent relationships and shared information between the classification heads to improve generalization ability and reduce overfitting.

[0114] Based on the same inventive concept, this application also provides a multi-task Mamba-CNN system for detecting hydrogen-blended natural gas pipeline leaks, used to implement the aforementioned multi-task Mamba-CNN method for detecting hydrogen-blended natural gas pipeline leaks. The solution provided by this system is similar to the implementation described in the above method; therefore, the specific limitations of one or more control module embodiments provided below can be found in the above-described limitations of the multi-task Mamba-CNN method for detecting hydrogen-blended natural gas pipeline leaks, and will not be repeated here. Specifically, Figure 4This is a schematic diagram of a hydrogen-blended natural gas pipeline leak detection system based on multi-task Mamba-CNN, as described in an embodiment of this application. Figure 4 As shown, the hydrogen-blended natural gas pipeline leak detection system based on multi-task Mamba-CNN includes:

[0115] The acquisition module 410 is used to acquire the audio data corresponding to the pipe;

[0116] Processing module 420 is used to process the audio data to obtain the log-Mel spectrum corresponding to the audio data;

[0117] The detection module 430 is used to process the log-Mel spectrum based on the leakage detection model to obtain the detection result; wherein, the leakage detection model is constructed based on a first normalization layer, a linear transformation layer, a state space sequence module, a convolutional neural network, and a multi-task classification head.

[0118] In some embodiments, the detection module 430 is further configured to normalize the log-Mel spectrum through the first normalization layer to obtain a normalized log-Mel spectrum.

[0119] The normalized log-Mel spectrum is projected into a low-dimensional latent space through the linear transformation layer to obtain the low-dimensional feature sequence corresponding to the normalized log-Mel spectrum.

[0120] The low-dimensional feature sequence is processed by the state space sequence module to obtain the enhanced feature sequence corresponding to the low-dimensional feature sequence;

[0121] The enhanced feature sequence is processed by the convolutional neural network to perform feature extraction and adaptive average pooling operations to obtain the high-level feature vector corresponding to the enhanced feature sequence.

[0122] The high-level feature vector is classified using the multi-task classification head to obtain the detection result.

[0123] In some embodiments, the state space sequence module includes a first state space sequence unit and a second state space sequence unit; the second state space sequence unit is stacked in series with the first state space sequence unit.

[0124] The detection module 430 is further configured to perform temporal modeling and normalization processing on the low-dimensional feature sequence through the first state space sequence unit to obtain the initial enhanced feature sequence corresponding to the low-dimensional feature sequence; and to perform temporal modeling and normalization processing on the initial enhanced feature sequence again through the second state space sequence unit to obtain the enhanced feature sequence.

[0125] In some embodiments, the first state space sequence unit includes a sequence modeling layer and a second normalization layer;

[0126] The detection module 430 is further configured to perform time-series modeling on the low-dimensional feature sequence through the sequence modeling layer to obtain an initial feature sequence; and to perform normalization processing on the initial feature sequence through the second normalization layer to obtain the first feature sequence.

[0127] In some embodiments, the convolutional neural network includes a plurality of convolutional blocks and an adaptive average pooling layer;

[0128] The detection module 430 is further configured to perform feature extraction processing on the enhanced feature sequence through multiple convolutional blocks to obtain an initial high-level feature vector; and to perform adaptive average pooling operation on the initial high-level feature vector through the adaptive average pooling layer to obtain the high-level feature vector.

[0129] In some embodiments, the plurality of convolutional blocks include a first convolutional block, a second convolutional block, and a third convolutional block, wherein the number of channels corresponding to the first convolutional block is less than the number of channels corresponding to the second convolutional block, and the number of channels corresponding to the second convolutional block is less than the number of channels corresponding to the third convolutional block.

[0130] The detection module 430 is further configured to perform feature extraction processing on the enhanced feature sequence through the first convolutional block to obtain a first-level feature; perform feature extraction processing on the first-level feature through the second convolutional block to obtain a second-level feature; and perform feature extraction processing on the second-level feature through the third convolutional block to obtain the initial high-level feature vector.

[0131] In some embodiments, the first convolutional block includes a depth-separable convolutional layer, a batch normalization layer, an activation layer, a pooling layer, and a dropout layer;

[0132] The detection module 430 is further configured to extract features from the enhanced feature sequence through the depthwise separable convolutional layer to obtain a first feature sequence; perform batch normalization processing on the first feature sequence through the batch normalization layer to obtain a second feature sequence; perform activation processing on the second feature sequence through the activation layer to obtain a third feature sequence; perform pooling processing on the third feature sequence through the pooling layer to obtain a fourth feature sequence; and perform regularization processing on the fourth feature sequence through the dropout layer to obtain the first-level feature.

[0133] In some embodiments, the detection module 430 is further configured to obtain a detection result based on the high-level feature vector using the multi-task classification head, including: classifying the high-level feature vector as a natural gas leak using the first classification head to obtain the leak detection result; classifying the high-level feature vector as a hydrogen mixture ratio using the second classification head to obtain the hydrogen mixture ratio classification result; and classifying the high-level feature vector as a pressure level using the third classification head to obtain the pressure level classification result.

[0134] In some embodiments, the processing module 420 is further configured to perform frame-segmentation processing on the audio data and apply a window function to each frame to obtain multiple frame data corresponding to the audio data; perform short-time Fourier transform on the multiple frame data respectively to obtain multiple spectra corresponding to the multiple frame data; determine multiple power spectra corresponding to the multiple spectra, and perform nonlinear mapping and weighted summation on the multiple power spectra through a Mel filter to obtain a Mel spectrogram; and perform logarithmic scaling on the energy value of each frequency band in the Mel spectrogram to obtain the logarithmic Mel spectrogram.

[0135] like Figure 5 As shown in the illustration, a control device provided in this application embodiment may include: a processor 510, a communication interface 520, a memory 530, and a communication bus 540, wherein the processor 510, the communication interface 520, and the memory 530 communicate with each other through the communication bus 540. The processor 510 may call logical instructions in the memory 530 to execute the methods described above.

[0136] Furthermore, the logical instructions in the aforementioned memory 530 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the multi-task Mamba-CNN hydrogen-blended natural gas pipeline leak detection method described in various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0137] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the methods described above.

[0138] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0139] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0140] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of the invention. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of the invention, the sequence numbers of the above-described processes do not imply a sequential order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the invention. The sequence numbers of the above-described embodiments of the invention are merely descriptive and do not represent the superiority or inferiority of the embodiments.

[0141] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0142] In the several embodiments provided by this invention, it should be understood that the disclosed methods can be implemented in other ways. The methods disclosed in the several method embodiments provided by this invention can be arbitrarily combined without conflict to obtain new method embodiments. The features disclosed in the several method embodiments provided by this invention can be arbitrarily combined without conflict to obtain new method embodiments.

[0143] The above description is merely an embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for detecting leaks in hydrogen-blended natural gas pipelines based on multi-task Mamba-CNN, characterized in that, include: Obtain the audio data corresponding to the pipe; The audio data is processed to obtain the corresponding log-Mel spectrum. The log-Mel spectrum is processed based on the leak detection model to obtain the detection results; wherein, the leak detection model is constructed based on a first normalization layer, a linear transformation layer, a state space sequence module, a convolutional neural network, and a multi-task classification head, and the detection results include leak detection results, hydrogen mixing ratio classification results, and pressure level classification results; The process of processing the log-Mel spectrum based on the leakage detection model to obtain the detection result includes: The log-Mel spectrum is normalized by the first normalization layer to obtain the normalized log-Mel spectrum. The normalized log-Mel spectrum is projected into a low-dimensional latent space through the linear transformation layer to obtain the low-dimensional feature sequence corresponding to the normalized log-Mel spectrum. The state space sequence module processes the low-dimensional feature sequence to obtain the enhanced feature sequence corresponding to the low-dimensional feature sequence. The enhanced feature sequence is processed by the convolutional neural network to perform feature extraction and adaptive average pooling operations to obtain the high-level feature vector corresponding to the enhanced feature sequence. The high-level feature vector is classified using the multi-task classification head to obtain the detection result.

2. The method according to claim 1, characterized in that, The state space sequence module includes a first state space sequence unit and a second state space sequence unit; the second state space sequence unit is stacked in series with the first state space sequence unit. The step of processing the low-dimensional feature sequence through the state space sequence module to obtain the enhanced feature sequence corresponding to the low-dimensional feature sequence includes: The low-dimensional feature sequence is temporally modeled and normalized using the first state space sequence unit to obtain the initial enhanced feature sequence corresponding to the low-dimensional feature sequence. The initial enhanced feature sequence is further subjected to temporal modeling and normalization processing by the second state space sequence unit to obtain the enhanced feature sequence.

3. The method according to claim 2, characterized in that, The first state-space sequence unit includes a sequence modeling layer and a second normalization layer; The step of performing temporal modeling and normalization on the low-dimensional feature sequence using the first state space sequence unit to obtain the initial enhanced feature sequence corresponding to the low-dimensional feature sequence includes: The sequence modeling layer is used to perform time-series modeling on the low-dimensional feature sequence to obtain the initial feature sequence. The initial enhanced feature sequence is obtained by normalizing the initial feature sequence through the second normalization layer.

4. The method according to claim 1, characterized in that, The convolutional neural network includes multiple convolutional blocks and an adaptive average pooling layer; The step of performing feature extraction and adaptive average pooling operations on the enhanced feature sequence through the convolutional neural network to obtain the high-level feature vector corresponding to the enhanced feature sequence includes: The enhanced feature sequence is processed by multiple convolutional blocks to obtain an initial high-level feature vector. The high-level feature vector is obtained by performing adaptive average pooling on the initial high-level feature vector through the adaptive average pooling layer.

5. The method according to claim 4, characterized in that, The plurality of convolutional blocks include a first convolutional block, a second convolutional block, and a third convolutional block, wherein the number of channels corresponding to the first convolutional block is less than the number of channels corresponding to the second convolutional block, and the number of channels corresponding to the second convolutional block is less than the number of channels corresponding to the third convolutional block; The step of performing feature extraction processing on the enhanced feature sequence through multiple convolutional blocks to obtain an initial high-level feature vector includes: The enhanced feature sequence is processed by the first convolutional block to obtain the first-level features; The second convolutional block is used to extract features from the first-level features to obtain the second-level features. The second-level features are extracted using the third convolutional block to obtain the initial high-level feature vector.

6. The method according to claim 5, characterized in that, The first convolutional block includes a depthwise separable convolutional layer, a batch normalization layer, an activation layer, a pooling layer, and a dropout layer; The step of performing feature extraction processing on the enhanced feature sequence through the first convolutional block to obtain first-level features includes: The enhanced feature sequence is extracted by the depthwise separable convolutional layer to obtain the first feature sequence. The first feature sequence is batch normalized through the batch normalization layer to obtain the second feature sequence; The second feature sequence is activated by the activation layer to obtain the third feature sequence; The third feature sequence is pooled using the pooling layer to obtain the fourth feature sequence; The fourth feature sequence is regularized using the discard layer to obtain the first-level feature.

7. The method according to claim 1, characterized in that, The multi-task classification head includes a first classification head, a second classification head, and a third classification head; The process of classifying the high-level feature vector using the multi-task classification head to obtain detection results includes: The leak detection result is obtained by classifying the high-rise feature vector using the first classification head; The hydrogen mixing ratio is classified by the second classification head on the high-level feature vector to obtain the hydrogen mixing ratio classification result; The pressure level classification result is obtained by classifying the high-level feature vector using the third classification head.

8. The method according to claim 1, characterized in that, The process of processing the audio data to obtain the corresponding log-Melbourne spectrogram includes: The audio data is segmented into frames, and a window function is applied to each frame to obtain multiple frame data corresponding to the audio data. Perform short-time Fourier transform on each of the multiple frame data to obtain multiple spectra corresponding to the multiple frame data; Multiple power spectra corresponding to multiple spectra are determined, and the multiple power spectra are nonlinearly mapped and weighted summed using a Mel filter to obtain a Mel spectrum. The logarithmic Mel spectrogram is obtained by logarithmically scaling the energy value of each frequency band in the Mel spectrogram.

9. A hydrogen-blended natural gas pipeline leak detection system based on multi-task Mamba-CNN according to any one of claims 1-8, characterized in that, The system includes: The acquisition module is used to acquire the audio data corresponding to the pipe. The processing module processes the audio data to obtain a log-Mel spectrum corresponding to the audio data; the detection module processes the log-Mel spectrum based on a leak detection model to obtain a detection result; wherein the leak detection model is constructed based on a first normalization layer, a linear transformation layer, a state space sequence module, a convolutional neural network, and a multi-task classification head, and the detection result includes a leak detection result, a hydrogen mixing ratio classification result, and a pressure level classification result; the module further normalizes the log-Mel spectrum through the first normalization layer to obtain a normalized log-Mel spectrum; projects the normalized log-Mel spectrum into a low-dimensional latent space through the linear transformation layer to obtain a low-dimensional feature sequence corresponding to the normalized log-Mel spectrum; processes the low-dimensional feature sequence through the state space sequence module to obtain an enhanced feature sequence corresponding to the low-dimensional feature sequence; performs feature extraction and adaptive average pooling operations on the enhanced feature sequence through the convolutional neural network to obtain a high-level feature vector corresponding to the enhanced feature sequence; and classifies the high-level feature vector through the multi-task classification head to obtain a detection result.

Images