A multi-modal fusion natural gas pipeline defect diagnosis method and system and storage medium

Abstract

The application discloses a kind of multi-modal fusion's natural gas pipeline defect diagnosis method and system and storage medium, it is related to natural gas pipeline leak detection field, to solve the problem of lack of comprehensive, efficient, high-precision detection method for internal and external defects of natural gas pipeline in prior art.Technical points: the application is based on image data and ultrasonic data respectively to extract features of internal defects of natural gas pipeline;Pipeline surface detection network based on YOLO is used to detect surface defects of pipeline, the SPD layer of network is used to convert the spatial dimension of feature map into depth dimension, sub-feature map is obtained by dividing from the specific area of original feature map, and the original feature map is down-sampled using scaling factor, non-step convolution layer is used to extract features without reducing the size of feature map;SPP module is used to capture feature information of different scales through multi-scale pooling operation, Elan module is used for multi-scale feature fusion, to realize the detection of pipeline surface defects.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of natural gas pipeline leak detection, in particular to a multi-modal fusion natural gas pipeline defect diagnosis method and system and a storage medium. BACKGROUND

[0002] Oil and gas pipelines are important infrastructure for energy transportation. During their long-term service, they may be affected by factors such as foundation settlement, corrosion, aging, mechanical damage, and environmental changes, which may cause structural deformation, material fatigue, and even failure, and thus lead to major safety accidents. In order to ensure the safe and stable operation of oil and gas pipelines, a high-precision and real-time pipeline defect detection system needs to be built.

[0003] Natural gas pipelines not only have internal damage but also have external damage during their long-term service. For internal damage, existing technologies often use ultrasonic flaw detection devices to determine whether there are internal defects based on human experience. This method is greatly affected by subjective factors, and it is often difficult to achieve high recognition accuracy using deep learning algorithms for defect recognition. Currently, external defect recognition of pipelines often requires manual inspection one by one. Using visual algorithms to detect defects on the surface can bring more convenience, but in complex tasks with low resolution or small objects, there are often problems of low feature extraction efficiency and insufficient accuracy. SUMMARY

[0004] The technical problem to be solved by the present application is:

[0005] There is a lack of comprehensive, efficient, and high-precision detection methods for internal and external defects of natural gas pipelines in the prior art.

[0006] The technical solution adopted by the present application to solve the above technical problem is:

[0007] The present application provides a multi-modal fusion natural gas pipeline defect diagnosis method, comprising the following steps:

[0008] Step S100, detecting internal ultrasonic data of the natural gas pipeline by an ultrasonic flaw detection device, and collecting surface image data of the natural gas pipeline;

[0009] Step S200, converting the internal ultrasonic data of the pipeline into two-dimensional image data, and using a double-branch detection model to extract features of internal defects of the natural gas pipeline based on the image data and the ultrasonic data respectively;

[0010] Step S300: Construct a YOLO-based pipeline surface detection network. The backbone network of the model introduces the SPD-Conv module and the SPP-Elan module. The SPD-Conv module consists of an SPD layer and a non-stretch convolutional layer. The SPD layer is used to convert the spatial dimension of the feature map into the depth dimension, divide the original feature map into sub-feature maps from specific regions, and downsample the original feature map using a scaling factor. The non-stretch convolutional layer is used to extract features without reducing the size of the feature map, so as to retain more granular information. The SPP-Elan module consists of an SPP module and an Elan module. The SPP module is used to capture feature information at different scales through multi-scale pooling operations. The Elan module is based on an efficient layer aggregation mechanism and is used for multi-scale feature fusion.

[0011] Step S400: Use a pipe surface inspection network to detect defects on the pipe surface.

[0012] Furthermore, the ultrasonic flaw detection device in step S100 includes a transmitting circuit module that excites the ultrasonic probe to work. The transmitting circuit module includes a high-voltage circuit module and a spike pulse conversion circuit. The high-voltage circuit module uses a DC voltage boosting module to convert the input voltage into a high voltage. The spike pulse conversion circuit uses the charging and discharging characteristics of resistors and capacitors to convert the PWM wave into a spike pulse signal. At the same time, the ultrasonic flaw detection device uses a limiting circuit based on diode characteristics as the receiving circuit.

[0013] Furthermore, the method for converting the ultrasonic data inside the pipe into two-dimensional image data in step S200 includes the following process: first, constructing a Gram matrix by the pairwise inner product of any N vectors in Euclidean space;

[0014] Suppose a set of column vectors forms a matrix X, whose Gram matrix is ​​defined as follows: After unfolding, we get:

[0015]

[0016] We further obtain the following expression:

[0017]

[0018] Among them G represents the angle between vectors i and j, and G represents the correlation between vectors, which is expressed by the angle between the vectors.

[0019] Assume the pipeline signal has N time points, and the signal data at each time point is... The corresponding timestamp is Then the pipeline signal is converted to polar coordinates:

[0020]

[0021] By the angle between vectors and The correlation between data points is represented, and the Gram matrix is formed as:

[0022]

[0023] The final inner product expression is: Each element in the Gram matrix is mapped to a pixel point, thereby forming an image.

[0024] Further, the step 200 further comprises adopting a BILSTM network to perform noise reduction processing on the collected pipeline ultrasonic signal data.

[0025] Further, the double-branch detection model in the step 200 is respectively based on a CNN network and an LSTM network, the CNN network is used for two-dimensional feature extraction and identification of image data, and the LSTM network is used for one-dimensional feature extraction and identification of pipeline internal ultrasonic data.

[0026] Further, the function implementation process of the SPD-Conv module in the step 300 is as follows: for any intermediate feature map of size SxSxC1 The SPD layer first divides it into multiple sub-feature maps according to the set scale size, the size of each sub-feature map is , wherein respectively represent the position of the sub-feature map in the original feature map, any generated sub-feature map can be divided, and then the sub-feature maps are spliced along the channel dimension to form a new feature map ; a non-strided convolution layer is used to perform convolution operation on , a 3x3 convolution kernel is used, the output channel number is , wherein , and finally a feature map of size is obtained.

[0027] Further, the function implementation process of the SPP-Elan module in the step 300 is as follows: for a feature map of size , the SPP module performs pooling operation on the feature map through different scale pooling windows to generate multiple scale feature maps , the size is ​The feature maps are spliced along the channel dimension to form a new feature map ; the ELAN module divides the feature map into a plurality of sub-feature maps , each sub-feature map has a size of , wherein is the number of sub-feature maps after division; a 1x1 convolution is applied to each sub-feature map for dimension reduction to generate an intermediate feature map , and a 3x3 convolution kernel is used for feature extraction to generate a final feature map ; the sub-feature maps are spliced along the channel dimension to form a final feature map .

[0028] The application provides a multi-modal fusion natural gas pipeline defect diagnosis system, which has program modules corresponding to the steps of the method described in any of the above technical solutions, and executes the steps in the multi-modal fusion natural gas pipeline defect diagnosis method described above when running.

[0029] The application also provides a computer-readable storage medium storing a computer program configured to implement the steps in the multi-modal fusion natural gas pipeline defect diagnosis method described in any of the above technical solutions when called by a processor.

[0030] Compared with the prior art, the application has the following advantages:

[0031] The application simultaneously detects internal and external defects of the pipeline to achieve comprehensive detection of natural gas pipeline defects. Meanwhile, the SPD-Conv module and the SPP-Elan module are introduced into the pipeline surface detection network based on YOLO constructed by the application, the SPD-Conv module not only retains all information in the feature map, but also reduces the number of channels through a non-cross-row convolution layer, thereby improving the efficiency of feature extraction. The SPP-Elan module captures feature information of different scales through a multi-scale pooling operation, significantly enhances the detection ability of the model for multi-scale targets, and further enhances the convergence performance and learning ability of the network through an efficient layer aggregation mechanism, thereby improving the efficiency of feature extraction and the overall performance of the model while maintaining high precision, and achieving efficient and high-precision natural gas pipeline defect detection. BRIEF DESCRIPTION OF DRAWINGS

[0032] Figure 1 It is a circuit schematic diagram of the ultrasonic flaw detection device in the embodiment of the application;

[0033] Figure 2 It is a physical diagram of the ultrasonic flaw detection device in the embodiment of the application;

[0034] Figure 3 The ultrasonic defect signal BILSTM denoising processing diagram in the embodiment of the application;

[0035] Figure 4 The two-dimensional image using Gram angle field conversion in the embodiment of the application;

[0036] Figure 5 The CNN+LSTM double branch feature extraction diagram in the embodiment of the application;

[0037] Figure 6 The space-depth conversion convolution module schematic diagram in the embodiment of the application;

[0038] Figure 7 The SPP module schematic diagram in the pyramid feature aggregation module in the embodiment of the application;

[0039] Figure 8 The ELAN module schematic diagram in the pyramid feature aggregation module in the embodiment of the application;

[0040] Figure 9 The subject architecture of the multi-scale feature fusion detection network model for visual detection of natural gas pipeline defects in the embodiment of the application;

[0041] Figure 10 The technical roadmap of the natural gas pipeline defect detection system based on ultrasonic and visual detection in the embodiment of the application;

[0042] Figure 11 The natural gas pipeline defect diagnosis system interface of the multi-modal fusion in the embodiment of the application. DETAILED DESCRIPTION

[0043] In order for those skilled in the art to better understand the present application, exemplary embodiments or examples of the present application will be described below in conjunction with the accompanying drawings. Obviously, the described embodiments or examples are only part of the embodiments or examples of the present application, not all. Based on the embodiments or examples in the present application, all other embodiments or examples obtained by those skilled in the art without creative labor should fall within the scope of the present application.

[0044] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the specific embodiments of the present application will be described in detail below in conjunction with the accompanying drawings.

[0045] Specific implementation scheme one: the present application provides a multi-modal fusion natural gas pipeline defect diagnosis method, as shown in the following steps: Figure 10

[0046] ​Step S100, detecting internal ultrasonic wave data of the natural gas pipeline by the ultrasonic flaw detection device, and collecting surface image data of the natural gas pipeline;

[0047] Step S200, converting the internal ultrasonic wave data of the pipeline into two-dimensional image data, and adopting a double-branch detection model to respectively extract and identify features of internal defects of the natural gas pipeline based on the image data and the ultrasonic wave data;

[0048] Step S300, constructing a pipeline surface detection network based on YOLO, introducing an SPD-Conv module and an SPP-Elan module into a backbone network of the model, the SPD-Conv module being composed of an SPD layer and a non-step convolution layer, the SPD layer being used to convert a spatial dimension of a feature map into a depth dimension, sub-feature maps being obtained by dividing a specific region of the original feature map, and the original feature map being down-sampled by using a scaling factor, the non-step convolution layer being used to extract features without reducing the size of the feature map, so that more granularity information is retained, the SPP-Elan module being composed of an SPP module and an Elan module, the SPP module being used to capture feature information of different scales by a multi-scale pooling operation, and the Elan module being based on an efficient layer aggregation mechanism and being used to perform multi-scale feature fusion;

[0049] Step S400, detecting defects on the surface of the pipeline by using the pipeline surface detection network.

[0050] The SPD-Conv module of the embodiment can effectively increase the depth of the feature map without losing learning information, and extract features without reducing the size of the feature map, so that more granularity information is retained, and a richer feature basis is provided for subsequent feature fusion and target detection. The SPP-Elan module fully combines the advantages of the SPP and ELAN networks, so that the model performs better when processing complex scenes and multi-scale targets.

[0051] Specific embodiment two: in order to trigger the ultrasonic probe to work, the ultrasonic flaw detection device needs to be designed as Figures 1-2The transmitting and receiving circuits shown include a high-voltage circuit module and a spike pulse conversion circuit designed to excite the ultrasonic probe. The high-voltage circuit module uses a DC voltage booster to convert the input 15V voltage to 100V to provide high-voltage energy to the ultrasonic probe. Simultaneously, since the ultrasonic probe typically uses pulse excitation to generate the required ultrasonic signal, an STM32F407 system development board outputs a PWM wave with a repetition frequency of 2kHz and a pulse width of 600ns. Finally, the PWM wave is converted into a spike pulse signal through the charging and discharging characteristics of a resistor-capacitor (RC) circuit. These two circuits together constitute the transmitting circuit, generating a high-voltage spike pulse signal to activate the ultrasonic probe. Since the echo signal contains the high-voltage signal from the transmitting circuit, a limiting circuit using diode characteristics is designed to filter out the high-voltage signal, forming the receiving circuit. Other aspects of this implementation scheme are the same as in Specific Implementation Scheme 1.

[0052] Specific Implementation Scheme 3: The method for converting the ultrasonic data inside the pipe into two-dimensional image data in step S200 includes the following process:

[0053] like Figures 3-4 As shown, the acquired defect signal is stored in the host computer through the data storage function of the oscilloscope. The data is read and displayed using MATLAB programming. The defect signal contains various environmental noises, which are denoised using the BILSTM network model.

[0054] In Euclidean space, the pairwise inner products of any N vectors form a Gram matrix, defined as follows:

[0055]

[0056] Suppose a set of column vectors forms a matrix X, whose Gram matrix is ​​defined as follows: After unfolding, we get:

[0057]

[0058] As can be seen from the above formula, the Gram matrix is ​​a symmetric matrix, where the diagonal elements provide the characteristics of each vector, and the remaining elements provide relevant information between different vectors. Assuming that these column vectors are all unit vectors, we can further obtain the following expression:

[0059]

[0060] in G represents the angle between vectors i and j. G represents the correlation between the vectors, which is expressed by the angle between them.

[0061] For the ultrasonic signal data of the pipeline, it is converted into vector form through polar coordinate transformation: Assume there are N time points in the pipeline signal, and the signal data at each time point is... The corresponding timestamp is Then the pipeline signal is converted to polar coordinates:

[0062]

[0063] Through vectors and Angle between The correlation between data points is represented by the Gram matrix:

[0064]

[0065] The final inner product expression is: Each element in the Gram matrix is ​​mapped to a pixel, thus forming an image. This implementation scheme is otherwise identical to specific implementation scheme two.

[0066] In this implementation, a bidirectional mapping between data points and their temporal locations is achieved by converting pipeline ultrasonic data into polar coordinate encoding: data points (normalized data) are converted into polar angle representations, and timestamps are converted into polar axis representations, allowing the signal to be represented as a vector. For pipeline signals, the data at each time point can be considered as a vector, and the angle between any two vectors is calculated. The correlation is then expressed using angle definitions in the Gram matrix. This method preserves the temporal dependence and structural information of the original data while transforming it into an image format that can be effectively processed in the field of computer vision, enabling the detection of internal defects in dual-branch pipelines.

[0067] Specific implementation plan four: The dual-branch detection model mentioned in step 200 is based on CNN and LSTM networks respectively, such as... Figure 5 As shown, the CNN network is used for two-dimensional feature extraction of image data, and the LSTM network is used for one-dimensional feature extraction of ultrasonic data inside the pipe, enabling high-precision classification of defect signals. Other aspects of this implementation scheme are the same as in specific implementation scheme three.

[0068] Specific implementation scheme five: The functional implementation process of the SPD-Conv module mentioned in step 300 is as follows: For any intermediate feature map of size S×S×C1 The SPD layer first divides the slice into multiple sub-feature maps according to the set slice size. The size of each sub-feature map is ,in These represent the positions of the sub-feature maps in the original feature maps, respectively. Any generated sub-feature can be... integer division, and then concatenating these sub-feature maps along the channel dimension to form a new feature map with size , and the specific formula is: ; the non-strided convolution layer is used to perform convolution operation on , using a 3x3 convolution kernel, a stride of 1, and an output channel number of , wherein , to obtain a feature map with a size of , and the specific formula is: This design not only retains all the information in the feature map, but also reduces the number of channels through the non-strided convolution layer, thereby improving the efficiency of feature extraction. The other aspects of this embodiment are the same as Embodiment Four.

[0069] As shown in Figure 6 , given any feature Figure X , a sub-map is composed of all elements of , and any sub-feature map generated in this way can be proportionally cut. Therefore, each sub-map reduces the sample X to a scale factor of Figure 6 , which gives an example of a scale factor of 2, in which four sub-maps are obtained, each sub-map has a shape of , and the sample is reduced to 2 times. These sub-feature maps are concatenated along the channel dimension to obtain a feature map , which has a proportionally reduced spatial dimension and a proportionally increased channel dimension 2 . That is, the SPD converts the feature map into an intermediate feature map ’( 2 C1). By using non-strided convolution to retain as much discriminative feature information as possible, the model improves the ability of feature extraction in low-resolution image detection and small target recognition.

[0070] Embodiment Six: The function implementation process of the SPP-Elan module in step 300 is as follows: for a feature map with a size of , the SPP module performs pooling operation on the feature map by using different scale pooling windows to generate multiple scale feature maps with a size of , and concatenating these feature maps along the channel dimension to form a new feature map , size of wherein is the number of pooling windows. The formula is expressed as: , ; the ELAN module divides into multiple sub-feature maps , each sub-feature map has a size of wherein is the number of divided sub-feature maps; a 1x1 convolution is applied to each sub-feature map for dimension reduction to generate an intermediate feature map , and then a 3x3 convolution kernel is used for feature extraction to generate a final feature map ; these sub-feature maps are spliced along the channel dimension to form a final feature map , size of . The formula is expressed as: .

[0071] The other aspects of the present embodiment are the same as those of the fifth embodiment.

[0072] The SPP-Elan module in the present embodiment is composed of an SPP-net and an Elan module. The SPP-net introduces a spatial pyramid pooling (SPP) layer between the convolution layer and the fully connected layer, so that the fully connected layer can receive a fixed number of features. This structure can generate a fixed-length feature representation for images of any size or scale, so there is no need for pre-processing operations such as cropping or scaling of the input image.

[0073] As shown in Figure 7 , it is assumed that an image with a size of (w, h) is input, wherein w and h represent the width and height of the feature map, respectively. If 21 features are needed to be extracted, the feature map can be regionally divided by different scale division methods, and finally 21 bins are obtained. One feature is extracted from each bin, and a 21-dimensional feature vector is obtained.

[0074] For example, it is assumed that the size of the feature map input to the SPP layer after the convolution operation is 16x16, and a fixed-dimensional feature representation can be extracted by the above pyramid pooling method. As Figure 8As shown, the Elan module is composed of VoVNet and CSPNet, and introduces the strategy of "stacked computing blocks", which avoids the excessive use of transition layers by reasonably arranging the stacking mode of the computing blocks, thereby preventing the shortest gradient path of the entire network from becoming rapidly longer. At the same time, Elan also optimizes the gradient propagation path of the entire network, ensuring that the shortest gradient path of each layer can be effectively trained, while controlling the longest gradient path of the entire network. Not only improves the convergence performance and learning ability of the network, but also reduces the parameter amount and calculation amount, improves the inference speed, so that the deep network can still maintain good performance when increasing the depth.

[0075] The multi-modal fusion natural gas pipeline defect diagnosis method (algorithm) provided by the present application is the underlying technical core of the present application, and various products can be derived based on the algorithm.

[0076] Based on the method proposed by the present application, a multi-modal fusion natural gas pipeline defect diagnosis system is developed using a program language, as shown in the figure, the system has program modules corresponding to the steps of the above technical solutions, and executes the steps in the above multi-modal fusion natural gas pipeline defect diagnosis method when running. Figure 11

[0077] The computer program of the developed system (software) is stored on a computer readable storage medium, and the computer program is configured to realize the steps of the above multi-modal fusion natural gas pipeline defect diagnosis method when called by a processor. That is, the present application is materialized on a carrier to become a computer program product.

[0078] The various embodiments of the systems and techniques described herein can be implemented in digital electronic circuitry, integrated circuitry, specially designed ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0079] ​The computational programs (also referred to as programs, software, software applications, or code) of this invention include machine instructions of a programmable processor and can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, device, and / or apparatus (e.g., disk, optical disk, memory, programmable logic device PLD) for providing machine instructions and / or data to a programmable processor, including machine-readable media that receive machine instructions as machine-readable signals. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0080] The beneficial effects of the present invention will be described below with reference to specific embodiments.

[0081] Example 1

[0082] like Figure 9 As shown, this embodiment constructs a pipeline surface inspection network with YOLOv10 as the baseline. The SPD-Conv module and SPP-Elan module are introduced into the backbone network. It is compared with YOLOv8, YOLOv9, the baseline YOLOv10, and the subsequent YOLOv11 and YOLOv12 on the same pipeline defect dataset. The results of various evaluation indicators are shown in Table 1.

[0083] Table 1

[0084]

[0085] While the present invention has been disclosed above, its scope of protection is not limited thereto. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the present invention, and all such changes and modifications will fall within the scope of protection of the present invention.

Claims

1. A multimodal fusion method for diagnosing defects in natural gas pipelines, characterized in that: Includes the following steps: Step S100: Detect the ultrasonic data inside the natural gas pipeline using an ultrasonic flaw detection device and collect image data of the natural gas pipeline surface; Step S200: Convert the ultrasonic data inside the pipeline into two-dimensional image data, and use a dual-branch detection model to extract features of internal defects in the natural gas pipeline based on the image data and ultrasonic data respectively. Step S300: Construct a YOLO-based pipeline surface detection network. The backbone network of the model introduces the SPD-Conv module and the SPP-Elan module. The SPD-Conv module consists of an SPD layer and a non-stretch convolutional layer. The SPD layer is used to convert the spatial dimension of the feature map into the depth dimension, divide the original feature map into sub-feature maps from specific regions, and downsample the original feature map using a scaling factor. The non-stretch convolutional layer is used to extract features without reducing the size of the feature map, so as to retain more granular information. The SPP-Elan module consists of an SPP module and an Elan module. The SPP module is used to capture feature information at different scales through multi-scale pooling operations. The Elan module is based on an efficient layer aggregation mechanism and is used for multi-scale feature fusion. Step S400: Use a pipe surface inspection network to detect defects on the pipe surface; The ultrasonic flaw detection device in step S100 includes a transmitting circuit module that excites the ultrasonic probe to work. The transmitting circuit module includes a high-voltage circuit module and a spike pulse conversion circuit. The high-voltage circuit module uses a DC voltage boosting module to convert the input voltage into a high voltage. The spike pulse conversion circuit uses the charging and discharging characteristics of resistors and capacitors to convert the PWM wave into a spike pulse signal. At the same time, the ultrasonic flaw detection device uses a limiting circuit based on diode characteristics as the receiving circuit. The method for converting ultrasonic data inside the pipe into two-dimensional image data in step S200 includes the following process: First, the inner product of any N vectors in Euclidean space is used to construct a Gram matrix; Suppose a set of column vectors forms a matrix X, whose Gram matrix is ​​defined as follows: After unfolding, we get: We further obtain the following expression: Among them G represents the angle between vectors i and j, and G represents the correlation between vectors, which is expressed by the angle between the vectors. Assume the pipeline signal has N time points, and the signal data at each time point is... The corresponding timestamp is Then the pipeline signal is converted to polar coordinates: Through vectors and Angle between The correlation between data points is represented by the Gram matrix: The final inner product expression is: Each element in the Gram matrix is ​​mapped to a pixel, thus forming an image.

2. The multimodal fusion method for diagnosing defects in natural gas pipelines according to claim 1, characterized in that: Step 200 also includes using a BILSTM network to perform noise reduction processing on the acquired pipeline ultrasonic signal data.

3. The multimodal fusion method for natural gas pipeline defect diagnosis according to claim 2, characterized in that: The dual-branch detection model in step 200 is based on CNN and LSTM networks, respectively. The CNN network is used to extract and recognize two-dimensional features from image data, and the LSTM network is used to extract and recognize one-dimensional features from ultrasonic data inside the pipe.

4. The multimodal fusion method for natural gas pipeline defect diagnosis according to claim 3, characterized in that: The functional implementation process of the SPD-Conv module in step 300 is as follows: for any intermediate feature map of size S×S×C1 The SPD layer first divides the slice into multiple sub-feature maps according to the set slice scale. The size of each sub-feature map is ,in These represent the positions of the sub-feature maps within the original feature maps. Any generated sub-feature map can be... Divide by integer, then concatenate these sub-feature maps along the channel dimension to form a new feature map. Using non-stride convolutional layers Perform convolution operations using a 3×3 convolution kernel, with the number of output channels being... ,in The final size is Feature Mapping .

5. The multimodal fusion method for natural gas pipeline defect diagnosis according to claim 4, characterized in that: The functional implementation process of the SPP-Elan module in step 300 is as follows: for a size of Feature Mapping The SPP module uses pooling windows of different scales. Pooling operations are performed on the feature maps to generate feature maps at multiple scales. Size is These feature maps are concatenated along the channel dimension to form a new feature map. The ELAN module will Segmented into multiple sub-feature maps The size of each sub-feature map is ,in It is the number of sub-feature maps in the segmentation; for each sub-feature map Dimensionality reduction is performed using 1×1 convolutions to generate intermediate feature maps. Then, a 3×3 convolution kernel is used for feature extraction to generate the final feature map. ; These sub-feature maps are concatenated along the channel dimension to form the final feature map. .

6. A multimodal fusion-based natural gas pipeline defect diagnosis system, characterized in that: The system has a program module corresponding to the steps of the method described in any one of claims 1 to 5, and executes the steps in the above-described multimodal fusion natural gas pipeline defect diagnosis method when running.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program configured to implement, when invoked by a processor, the steps of the multimodal fusion method for diagnosing defects in natural gas pipelines according to any one of claims 1 to 5.

Images