Method, device and system for identifying a gas pipeline leak

Abstract

The application provides a method, device and system for identifying gas pipeline leakage, wherein the method comprises: acquiring a gas pipeline image to be identified; and using a neural network model constructed and trained based on a backbone feature extraction network, a feature fusion network and a classification prediction network to identify whether a gas leakage point exists in the gas pipeline image to be identified. The application solves the technical problem that the prior art cannot identify gas leakage when the leakage size of the gas pipeline is small.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, in particular to a method, device and system for identifying gas pipeline leakage. BACKGROUND

[0002] As an important carrier for natural gas transportation, gas pipeline is not only efficient and convenient, but also can greatly reduce the cost of gas transportation. Pipeline leakage not only causes economic loss and environmental pollution, but also poses a great threat to people's life and property safety. Therefore, buried gas pipeline must have a good working condition. However, most of the city gas pipelines are buried underground, and the buried time of buried gas pipelines is different, and most of the city buried gas pipelines in China are in the late design life, and the scale of the pipeline to be repaired is large, and external force, corrosion, cracking and other factors can cause pipeline leakage. Therefore, it is necessary to regularly maintain the city buried gas pipeline.

[0003] At present, the methods for identifying buried gas pipeline leakage mainly include two types: one is direct observation method, that is, the maintenance personnel directly enter the pipeline to observe whether the pipeline leaks. This method not only requires the maintenance personnel to keep focused at all times to avoid missing due to fatigue, but also depends on the subjective judgment of the staff, and the identification result is subjective. The second type is indirect identification method, such as negative pressure wave detection method and wavelet detection method, but when the leakage is small, this method cannot identify the existence of leakage. The inner diameter of DN100 pipeline is about 95mm, and the staff cannot enter the pipeline for maintenance, so excavation method must be used for maintenance. However, large-scale excavation is time-consuming and labor-intensive, and the cost is very high for pipelines that need to be frequently maintained. The first method is not suitable, and the second method cannot identify when the leakage size is small.

[0004] In view of the above problems, no effective solution has been proposed so far. SUMMARY

[0005] The embodiments of the present application provide a method, device and system for identifying gas pipeline leakage, to at least solve the technical problem that the prior art cannot identify gas leakage when the leakage size of the gas pipeline is small.

[0006] According to an aspect of an embodiment of the present application, a method for identifying gas pipeline leakage is provided, comprising: acquiring a gas pipeline image to be identified; and using a neural network model constructed and trained based on a backbone feature extraction network, a feature fusion network and a classification prediction network to identify whether there is a gas leakage point in the gas pipeline image to be identified.

[0007] According to another aspect of the embodiments of the present application, a device for identifying a gas pipeline leakage is also provided, comprising: an acquisition module configured to acquire a gas pipeline image to be identified; and an identification module configured to identify whether a gas leakage point exists in the gas pipeline in the gas pipeline image to be identified by using a neural network model constructed and trained in advance based on a backbone feature extraction network, a feature fusion network, and a classification prediction network.

[0008] According to still another aspect of the embodiments of the present application, a system for identifying a gas pipeline leakage is also provided, comprising: a gas pipeline; an image acquisition device configured to acquire an image of the gas pipeline by a camera device as a gas pipeline image to be identified; and the device for identifying a gas pipeline leakage as described above configured to identify whether a gas leakage point exists in the gas pipeline in the gas pipeline image to be identified.

[0009] In the embodiments of the present application, a gas pipeline image to be identified is acquired; and a neural network model constructed and trained in advance based on a backbone feature extraction network, a feature fusion network, and a classification prediction network is used to identify whether a gas leakage point exists in the gas pipeline in the gas pipeline image to be identified, thereby solving the technical problem that the prior art cannot identify a gas leakage when the size of a gas pipeline leakage is small. BRIEF DESCRIPTION OF DRAWINGS

[0010] The drawings accompanying the specification of the present application form a part thereof, serve to provide further understanding of the present application, and together with the specification explain the present application. The use of these drawings is to explain the present application and is not intended to limit the present application. In the drawings:

[0011] Figure 1 is a flowchart of a method for identifying a gas pipeline leakage according to an embodiment of the present application;

[0012] Figure 2 is a flowchart of another method for identifying a gas pipeline leakage according to an embodiment of the present application;

[0013] Figure 3 is a flowchart of a method for constructing an image data set according to an embodiment of the present application;

[0014] Figure 4 is a flowchart of a method for constructing a network model for identifying a city buried gas pipeline leakage according to an embodiment of the present application;

[0015] Figure 5 is a structure diagram of an algorithm for identifying a city buried gas pipeline leakage according to an embodiment of the present application;

[0016] Figure 6 is a structure diagram of a Focus module according to an embodiment of the present application;

[0017] Figure 7 is a structural schematic diagram of a CBL module according to an embodiment of the present application;

[0018] Figure 8 is a structural schematic diagram of a CSP module according to an embodiment of the present application;

[0019] Figure 9 is a structural schematic diagram of a SPP module according to an embodiment of the present application;

[0020] Figure 10 is a structural schematic diagram of a device for identifying a gas pipeline leakage according to an embodiment of the present application;

[0021] Figure 11 shows a structural schematic diagram of an electronic device suitable for implementing embodiments of the present disclosure. DETAILED DESCRIPTION

[0022] It should be noted that the embodiments and features of the embodiments in the present application can be combined with each other without conflict. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0023] It should be noted that the terms used herein are only intended to describe specific embodiments and are not intended to limit exemplary embodiments according to the present application. As used herein, the singular form is intended to include the plural form, unless the context clearly indicates otherwise, and it should also be understood that the terms "comprise" and / or "include" as used in the specification indicate the presence of features, steps, operations, devices, components and / or combinations thereof.

[0024] Unless specifically stated otherwise, the relative arrangements of the components and steps illustrated in these embodiments and the numerical expressions and values set forth herein are not meant to limit the scope of the present application. It should also be understood that the size of the various parts shown in the figures can not be to scale and that the drawings are not intended as precise illustrations of the instruments, methods and devices described herein. The techniques, methods and devices known to those of ordinary skill in the relevant art can not be discussed in detail, but should be considered as part of the description of the application. In all examples shown and discussed herein, any specific value should be interpreted as merely exemplary and not as a limitation. Thus, other examples of the exemplary embodiments can have different values. It should be noted that like reference numerals and letters in the following drawings represent similar items, and thus, once an item is defined in one drawing, it need not be discussed further in subsequent drawings.

[0025] Embodiment 1

[0026] In view of the deficiencies of existing gas pipeline leakage identification, the embodiment of the present application provides a kind of city buried gas pipeline small leakage image identification method, for accurately identifying whether city buried gas pipeline has leakage, provides technical reference for repair staff to identify buried gas pipeline leakage.

[0027] Figure 1 It is the method for identifying gas pipeline leakage according to the embodiment of the present application, as shown in Figure 1 The method comprises the following steps:

[0028] Step S102, obtain the image of the gas pipeline to be identified.

[0029] The parameters of the city buried gas pipeline to be identified in the embodiment of the present application can be: the material of the pipeline is PE pipeline; the specification of the pipeline is DN100 pipeline; the minimum size of the pipeline leakage reaches 1mm.

[0030] Step S104, using the neural network model constructed and trained based on the backbone feature extraction network, the feature fusion network and the classification prediction network in advance, to identify whether there is a gas leakage point in the image of the gas pipeline to be identified.

[0031] 1) Collect city buried gas pipeline leakage images as a data set.

[0032] Data augmentation is performed on the data set to expand the number of images, and the leakage points are labeled on the images containing the leakage points to form labels, and the city buried gas pipeline leakage image data set is constructed, and is divided into training set and test set according to 8:2.

[0033] Specifically, first, use CCD camera as shooting tool to collect city buried gas pipeline leakage images; then, adjust the size of all image data sets, normalize the images to a uniform size by using three linear interpolation scaling, set the training set picture size in the prepared input feature extraction network to 640x640x3; then, the normalized images are subjected to data enhancement processing to expand the number of data sets, and the training set images normalized to 640x640x3 are subjected to random rotation, translation, shearing, scaling, hue, saturation, brightness and other transformations; then, use labelimg image labeling tool to label the leakage points on the images containing the leakage points to form labels; finally, the image data set is divided into training set and test set according to 8:2 by using random division method.

[0034] 2) Construct a neural network model.

[0035] A neural network model (also referred to as an urban buried gas pipeline image leakage identification model) composed of three parts, a backbone feature extraction network, a feature fusion network introducing a Bi-directional Feature Pyramid Network (BiFPN), and a classification prediction network adding a small target detection layer, is constructed. The backbone feature extraction network is constructed using Focus, CBL, Res, CSP, and SPP modules; the BiFPN structure is introduced as the feature fusion network; and the small target detection layer is constructed in the classification prediction network to realize the identification of small size leakage points in the urban buried gas pipeline image.

[0036] Specifically, the neural network model can be constructed by the following steps:

[0037] (1) The feature extraction network is composed of modules such as Foucs, CBL, CSP, SPP, etc.

[0038] Among them, the CBL module is composed of three parts: ordinary convolution Conv, standardization BN, and LeakyReLU activation function; two CBL modules are used as the backbone part and the residual edge to construct the Res module; the CBL module, the Res module, and the ordinary convolution Conv are used as the backbone part and the ordinary convolution Conv, and then the BN, LeakyRelu, and CBL module are used to construct the CSP structure; the SPP module is composed of one CBL module, four maximum pooling kernels for maximum pooling operation, and one CBL module.

[0039] The preliminary feature layer f4 introduced from the fourth CSP module (CSP 4) in the feature fusion network passes through the first CBL module (CBL 1) in the feature fusion network to obtain the first enhanced feature layer P1; the P1 is subjected to feature fusion operation with the preliminary feature layer f3 introduced from the third CSP module (CSP 3) of the backbone feature extraction network after up-sampling, and after the fusion, the second enhanced feature layer P2 is obtained through the fifth CSP module (CSP 5) and the second CBL module (CBL 2) of the backbone feature extraction network; the P2 is connected with the second preliminary feature layer f2 introduced from the second CSP module (CSP 2) of the backbone feature extraction network after up-sampling, and after the connection, the third enhanced feature layer P3 is obtained through the sixth CSP module (CSP 6) and the convolution layer; the P3 is connected with the first preliminary feature layer f1 introduced from the first CSP module (CSP 1) of the backbone feature extraction network after up-sampling, and after the connection, the fourth enhanced feature layer P4 is obtained through the seventh CSP module (CSP 7); the P4 is connected with the P3 and f2 through the fourth CBL module (CBL 4), and after passing through the eighth CSP module (CSP 8), the fifth enhanced feature layer P5 is obtained; the P5 is connected with P2 and f3 through the fifth CBL module (CBL 5), and after passing through the ninth CSP module (CSP 9), the sixth enhanced feature layer P6 is obtained; the P6 is connected with P1 through the sixth CBL module (CBL 6), and after passing through the tenth CSP module (CSP 10), the seventh enhanced feature layer P7 is obtained.

[0040] (2) The input image is subjected to slicing operation through the Focus structure, the image size is changed to 320x320x12, and then subjected to convolution operation with 32 convolution kernels to finally change to a feature map with a size of 320x320x32; then, the CSP module and the CBL module are used three times, and then the CBL module and the SPP module are used to complete the construction of the backbone feature extraction network.

[0041] (3) The feature fusion network introduces the BiFPN structure, adopts the cross-scale connection line to enhance the feature fusion capability, and improves the identification accuracy of the leakage point;

[0042] (4) The head classification prediction network constructs a small target detection layer to realize the identification of small size leakage points in the buried gas pipeline image, reduce the missed detection rate of the leakage point identification, and improve the detection rate.

[0043] Specifically, after feature fusion, 7 enhanced feature layers P1-P7 are obtained, and the enhanced feature layers P4, P5, P6 and P7 are input into a head classification prediction network with four different sizes of 160x160, 80x80, 40x40 and 20x20 for classification prediction, so as to realize identification of micro, small, medium and large size leakage points in the city buried gas pipeline leakage image.

[0044] 3) Train the neural network model.

[0045] Train the city buried gas pipeline leakage identification network model. Divide the image data set into a training set and a test set in a ratio of 8:2, input the training set into the network model for training, and a total of 300 rounds of training are performed.

[0046] 4) Input the gas pipeline image to be identified.

[0047] Send the test set image into the trained city buried gas pipeline leakage identification network model to identify the leakage points in the test set image and verify the leakage identification effect of the network model.

[0048] 5) Perform identification.

[0049] Use the trained city buried gas pipeline leakage identification network model to identify the collected city buried gas pipeline leakage image (i.e., the gas pipeline image to be identified), so as to determine whether the city buried gas pipeline has leaked, and complete the identification of the city buried gas pipeline leakage.

[0050] Compared with the prior art, the present application has the following beneficial effects:

[0051] (1) The city buried gas pipeline micro leakage image identification method provided by the present application can automatically identify whether there is a leakage point in the image after obtaining the city buried gas pipeline image, improve the leakage point identification efficiency, has high identification accuracy, and can provide technical reference for pipeline inspection.

[0052] (2) The city buried gas pipeline leakage identification network model of the present application includes three parts: a feature extraction network, an introduced BiPFN structure as a feature fusion network, and a classification prediction network for constructing a small target detection layer. After image feature extraction, feature fusion and classification prediction, the city buried gas pipeline leakage identification network model is built, the training set image is used to train the built city buried gas pipeline leakage identification network model, and the trained network model has high identification accuracy and low missed detection rate.

[0053] Example 2

[0054] The current mainstream city gas pipeline leakage identification technology includes direct observation method and indirect identification method. The direct observation method needs the maintenance personnel to enter the pipeline for manual identification, and the workers need to concentrate on work at all times to avoid missed detection due to fatigue, and the identification result depends on the working experience of the workers, which is subjective. The city gas pipeline involved in the present application is a DN100 small buried gas pipeline, and the inner diameter is only 95mm, so the direct observation method is invalid. The indirect identification method can realize indirect identification with the aid of equipment, but when the size of the leakage point is small, these methods cannot accurately identify, and the reliability is low.

[0055] Therefore, an urban buried gas pipeline micro-leakage image identification method is proposed in the embodiments of the present application. The method mainly uses an urban buried gas pipeline leakage identification network model constructed based on a feature extraction network, a feature fusion network and a classification prediction network to identify micro-leakage. First, an urban buried gas pipeline leakage image is collected to construct a data set; the training set is input into the urban buried gas pipeline leakage identification network model for training; and the trained urban buried gas pipeline leakage identification network model is used to realize leakage identification.

[0056] The method proposed in the present application can realize the identification of the leakage point of the DN100 urban buried gas pipeline, and the minimum size of the leakage point can reach 1mm.

[0057] Specifically, the method is as shown in Figure 2 includes the following steps:

[0058] Step S202, collect the urban buried gas pipeline leakage image, label the leakage point on the image as a label, construct an image data set, and divide the data set into a training set and a test set.

[0059] The image data set construction method of the embodiments of the present application is as shown in Figure 3 includes the following steps:

[0060] Step S2022, use a CCD camera to collect the urban buried gas pipeline internal leakage image.

[0061] The CCD camera is used to collect the urban buried gas pipeline internal leakage image, and the present embodiment has a total of 171 images.

[0062] Step S2024, normalize all images using trilinear interpolation.

[0063] The 171 images are normalized to 640x640x3 by using the trilinear interpolation method.

[0064] Step S2026, process the normalized image.

[0065] Because too few image datasets can cause the city buried gas pipeline leakage recognition network model of the city buried gas pipeline leakage recognition algorithm to overfit, the 171 images are expanded in number by using data enhancement, including random rotation, translation, shearing, scaling, hue, saturation, brightness, and the like. The expanded number of images is 1000.

[0066] Step S2028, the image is labeled.

[0067] The 1000 expanded images are labeled to make label data, and the construction of the image dataset is completed.

[0068] Step S2029, the dataset is divided into a training set and a test set.

[0069] The image dataset is divided into a training set and a test set in a random manner, wherein the training set accounts for 80%, a total of 800 images, and the test set accounts for 20%, a total of 200 images.

[0070] Step S204, the city buried gas pipeline leakage image recognition network model is constructed.

[0071] The city buried gas pipeline leakage image recognition network model construction method of the embodiment of the application is as shown in Figure 4 , and includes the following steps:

[0072] Step S2042, the feature extraction network is constructed.

[0073] Step S2044, the BiFPN structure is introduced into the feature fusion network.

[0074] Step S2046, the small target detection layer is constructed in the classification prediction network to meet the identification of micro-sized leakage points.

[0075] In the city buried gas pipeline leakage image recognition network model, the city buried gas pipeline leakage recognition algorithm structure diagram is as shown in Figure 5 , the YOLO algorithm structure is complex, but can be composed of modules such as a Focus module, a CBL module, a CSP module, and an SPP module as shown in Figures 6 to 9 .

[0076] The city buried gas pipeline leakage recognition algorithm includes the following steps:

[0077] 1) Construct the basic modules required for the city buried gas pipeline leakage recognition network model.

[0078] As shown in Figure 6As shown, the Focus module inputs a 640x640x3 image input into the Focus structure, uses slicing operation to divide the image into 4 feature layers, and then stacks the 4 feature layers to change the channel number of the input image to 4 times the original.

[0079] As shown in Figure 7 The CBL module is composed of a common convolution (Conv), a batch normalization (BN), and an activation function (Leaky ReLU).

[0080] As shown in Figure 8 The CSP module is composed of a main side and a secondary side. The main side is composed of 1 CBL module, a Res Unit, and 1 common convolution. The secondary side is composed of 1 common convolution. After the main side and the secondary side are fused, 1 batch normalization, an activation function, and a CBL module are connected. The Res Unit is also composed of two sides. The first side is composed of 2 CBL modules, and the second side is a residual side. Figure 8 Figures (a) and (b) in

[0081] As shown in Figure 9 In the SPP module, the feature map input into the module first passes through 1 CBL module, then is subjected to maximum pooling by 4 maximum pooling layers, and finally the result after maximum pooling passes through 1 CBL module.

[0082] 2) The input end inputs a 640x640x3 image into the feature extraction network.

[0083] 3) The Focus module performs slicing and stacking operations on the image, and changes the image to 320x320x12.

[0084] 4) The feature map passes through the CBL module, and the feature map is changed to 320x320x64.

[0085] 5) The feature map passes through 3 CBL modules+CSP modules, and then passes through a CBL module and an SPP module. After 4 operations, the feature map is changed to 160x160x128, 80x80x256, 40x40x512, and 20x20x1024, respectively.

[0086] 6) From each CSP module in the above feature extraction network, 4 feature layers f1, f2, f3, and f4 are drawn from front to back, respectively. The feature layers correspond to 160x160x128, 80x80x256, 40x40x512, and 20x20x1024, respectively.

[0087] 7) Feature fusion.

[0088] The BiFPN structure is introduced into the feature fusion network. The preliminary feature layer f4 is extracted from the CSP 4, and the enhanced feature layer P1 is obtained after passing through a CBL module. P1 is first up-sampled, then fused with the preliminary feature layer f3 extracted from the CSP 3, and after the fusion, the enhanced feature layer P2 is obtained after passing through a CSP module and a CBL module. P2 is first up-sampled, then fused with the preliminary feature layer f2 extracted from the CSP 2, and after the fusion, the enhanced feature layer P3 is obtained after passing through a CSP module and a CBL module. P3 is first up-sampled, then fused with the preliminary feature layer f1 extracted from the CSP 1, and after the fusion, the enhanced feature layer P4 is obtained after passing through a CSP module. In addition, the enhanced feature layer also needs to be fused again from top to bottom. Specifically, P4 is first passed through a CBL module, then fused with P3 and f2, and after the fusion, the enhanced feature layer P5 is obtained after passing through a CSP module. P5 is first passed through a CBL module, then fused with P2 and f3, and after the fusion, the enhanced feature layer P6 is obtained after passing through a CSP module. P6 is first passed through a CBL module, then fused with P1, and after the fusion, the enhanced feature layer P7 is obtained after passing through a CSP module.

[0089] The algorithm of feature fusion can also use the method in Example 3 below. For details, please refer to Example 3, which will not be repeated here.

[0090] The introduction of the BiFPN structure and the feature fusion network in this embodiment can bring the following beneficial effects:

[0091] 1) Feature enhancement: By using CSP modules and CBL modules to process the preliminary feature layers, enhanced feature layers P1-P7 are obtained. These enhanced feature layers have more rich semantic information and stronger expression ability, which can improve the detection and recognition performance of the model on target objects.

[0092] 2) Multi-scale feature fusion: Through up-sampling operation and fusion with preliminary feature layers of different levels, such as the fusion of P1 and f3, the fusion of P2 and f2, etc., different scale feature information can be effectively fused. This multi-scale feature fusion helps to improve the recognition ability of the model on different size targets and enhances the context understanding ability of the target.

[0093] 3) Top-down re-fusion: After feature enhancement, through top-down re-fusion operations such as the fusion of P4 and P3, f2, the fusion of P5 and P2, f3, and the like, the expression ability of the features and the richness of the semantic information can be further improved. This top-down re-fusion can effectively pass the semantic information of high-level features to low-level features, improving the multi-scale perception and detail understanding ability of the model to the target.

[0094] 8) After the feature fusion network with the BiFPN structure is fused, 7 enhanced feature layers P1-P7 are obtained. The enhanced feature layers P4, P5, P6, and P7 are respectively input into four different size head classification prediction networks of 160x160, 80x80, 40x40, and 20x20 to perform classification prediction, so as to realize the identification of micro, small, medium, and large size leakage points in the city buried gas pipeline leakage image, mark the position of the leakage point in the image, frame the leakage point, and give the confidence information of the target belonging to the leakage point.

[0095] Step S206, training the city buried gas pipeline leakage identification network model by using the training set in the constructed data set.

[0096] The city buried gas pipeline leakage identification network model is trained using the training set in the constructed data set, with initial random weights, a total of 640x640x3 input image size, a training batch size of 16, an SGD optimizer, a thread number of 8, a preheating training mode, and a total of 300 rounds of training.

[0097] Step S208, test the trained city buried gas pipeline leakage identification network model by using the test set.

[0098] The test set is input into the trained city buried gas pipeline leakage identification network model for classification prediction, and the identification ability of the network model is tested.

[0099] Step S209, identify the leakage point in the city buried gas pipeline leakage image by using the trained network model.

[0100] The city buried gas pipeline micro-leakage image identification method proposed in the embodiments of the present application has the following beneficial effects:

[0101] 1) Automatic identification: Compared with the direct observation method which requires manual entry into the pipeline for identification, this method realizes automatic leakage identification by using an image identification network model, reducing the labor input and labor intensity.

[0102] 2) High accuracy: The leakage identification network model constructed by the feature extraction network, the feature fusion network, and the classification prediction network can realize accurate identification of micro-leakage, and the minimum leakage point size can reach 1mm.

[0103] 3) Effective avoidance of subjectivity: Compared to direct observation method which relies on the experience and subjective judgment of workers, this method uses network model for identification, avoiding the uncertainty brought by subjectivity.

[0104] 4) Data set construction and data enhancement: The data set is constructed by collecting urban buried gas pipeline leakage images, and the image quantity is expanded by data enhancement, which improves the diversity and quantity of data samples, and helps the training of network model and the accuracy of leakage identification.

[0105] 5) Scalability: The method proposes a leakage identification network model construction method, which can identify different size leakage points according to needs, with high flexibility and scalability.

[0106] Embodiment 3

[0107] The embodiment of the application provides a feature fusion algorithm. The feature fusion algorithm is used to combine feature information of different levels or different sources to improve the expression ability and performance of the model. The feature fusion method usually includes weighted sum fusion, element-wise addition fusion, splicing fusion, and attention mechanism fusion.

[0108] In the weighted sum fusion, the feature maps are fused by weighted sum. Each feature map has a corresponding weight, and the fused feature map is obtained by multiplying each feature map by the corresponding weight and summing. The weight can be manually set according to the specific task or obtained by learning.

[0109] In the element-wise addition fusion, the feature maps are fused by element-wise addition. The pixel values at the same position are added to obtain the fused feature map. It is suitable for the case where the dimensions of the feature maps are the same.

[0110] In the splicing fusion, the feature maps are spliced according to the channel dimension to generate a deeper feature map. By connecting multiple feature maps together in the channel dimension, a richer feature representation is formed.

[0111] The attention mechanism fusion fuses the feature maps by learning weights or attention distribution to enable the network to adaptively focus on different features.

[0112] Suppose there are N input feature maps, each with a dimension of C, the input feature map can be represented as a matrix X ∈ R^{H×W×C}. First, perform linear transformation on the input feature map to generate query (Q), key (K), and value (V) matrices.

[0113] The input feature map is mapped to different feature spaces by linear transformation, resulting in: Q = X * W Q K = X * W K V = X * W V where W Q ∈ R CxC', W K ∈ R CxC', W V ∈ R CxC' are learnable weight matrices, and C' is the intermediate feature dimension.

[0114] Next, the output of the multi-head attention mechanism is calculated. Q, K, and V are split into multiple heads (usually H) and the attention weights are calculated separately:

[0115]

[0116] where Qi represents the query matrix of the i-th attention head, which measures the importance and similarity of features. Ki represents the key matrix of the i-th attention head, which calculates the similarity between features. Vi represents the value matrix of the i-th attention head, which contains the information of the input feature map. W Q is the weight matrix of the query matrix, which learns the mapping of features. W K is the weight matrix of the key matrix, which learns the mapping of features. W V is the weight matrix of the value matrix, which learns the mapping of features; b is the bias vector, which adjusts the bias of the attention weight; d k is the dimension of the query and key, usually the dimension of the attention head (C' / H). dv is the dimension of the value, which determines the dimension of the value matrix. W V ' is the projection weight matrix of the value, which multiplies the value matrix for projection; c is the bias term, which adjusts the output of the attention mechanism.

[0117] Then, the final feature fusion result is obtained by linear transformation. Specifically, the output of each attention head is concatenated and linearly transformed to obtain the final feature fusion result:

[0118] Attention = Concatenate (Attention_1, Attention_2,..., Attention_H)

[0119] Fused_features = Attention * W O

[0120] where Attention_i is the output of the i-th attention head, Concatenate represents the concatenation operation, and W O is a learnable weight matrix.

[0121] Finally, by adding residual connection and normalization operation, the fused features are combined with the original input features:

[0122] Output = LayerNorm (Fused_features + X)

[0123] where LayerNorm represents the normalization operation, and Output is the final output feature map.

[0124] This multi-head attention mechanism fusion method can fully utilize the mutual relationship between input feature maps, and adaptively fuse features by calculating attention weights, so that the network can better capture the dependency and importance between features, and improve the representation ability and performance of the model.

[0125] The attention mechanism fusion method provided by the embodiment has the following beneficial effects:

[0126] 1) Adaptability. Through learning weights or attention distribution, the attention mechanism can adaptively focus on different regions and channels in the input feature map, so as to extract more useful features for the current task. This adaptability can improve the expression ability and robustness of the network.

[0127] 2) Feature importance weighting. Attention weights weight the features, so that the network can selectively fuse according to the importance of the features. Important features will get higher weights, so they will have greater influence in the fused feature representation, improving the network's perception of key features.

[0128] 3) Context modeling. By calculating the similarity between features, the attention mechanism can consider the context information between input feature maps during fusion. This helps to capture global context, semantic association and spatial dependency, thereby improving the discriminability and expressiveness of the features.

[0129] 4) Multi-head mechanism. Applying the attention mechanism to multiple heads can learn multiple different feature representations, with each head focusing on a different feature subspace. The multi-head mechanism can capture richer feature information and enhance the network's ability to perceive and fuse features of different scales and levels.

[0130] 5) Nonlinear transformation: Linear transformation is used to map input feature maps to different feature spaces, introducing nonlinearity to the attention mechanism. This allows the network to learn more complex feature representations and relationships, improving the network's expressiveness and generalization performance.

[0131] Embodiment 4

[0132] The embodiment of the present application provides a device for identifying gas pipeline leakage, as shown in Figure 10 The device comprises an acquisition module 12 and an identification module 14.

[0133] The acquisition module 12 is configured to acquire a gas pipeline image to be identified; the identification module 14 is configured to identify whether a gas leakage point exists in the gas pipeline in the gas pipeline image to be identified by using a neural network model constructed and trained in advance based on a backbone feature extraction network, a feature fusion network, and a classification prediction network.

[0134] The embodiment of the present application provides a system for identifying gas pipeline leakage, which comprises a gas pipeline, an image acquisition device and a device for identifying gas pipeline leakage. The image acquisition device is configured to acquire an image of the gas pipeline by a camera device as a gas pipeline image to be identified; and the device for identifying gas pipeline leakage is configured to identify whether a gas leakage point exists in the gas pipeline in the gas pipeline image to be identified.

[0135] It should be noted that the device for identifying gas pipeline leakage provided in the above embodiment is only exemplified by the division of the above functional modules, and in actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device for identifying gas pipeline leakage provided in the above embodiment and the method for identifying gas pipeline leakage provided in the embodiment belong to the same concept, and the specific implementation process is described in detail in the method embodiment, which will not be repeated here.

[0136] Embodiment 5

[0137] Figure 11 A structural schematic diagram of an electronic device suitable for implementing the embodiments of the present disclosure is shown. It should be noted that Figure 11 The electronic device shown is only an example and should not impose any limitation on the functions and use range of the embodiments of the present disclosure.

[0138] As Figure 11 shown, the electronic device includes a central processing unit (CPU) 1001, which can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 1002 or programs loaded from a storage portion 1008 into a random access memory (RAM) 1003. In the RAM 1003, various programs and data required for system operation are also stored. The CPU 1001, the ROM 1002, and the RAM 1003 are connected to each other through a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0139] The following components are connected to the I / O interface 1005: an input part 1006 including a keyboard, a mouse, etc.; an output part 1007 including a display such as a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage part 1008 including a hard disk, etc.; and a communication part 1009 including a network interface card such as a LAN card, a modem, etc. The communication part 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to the I / O interface 1005 as necessary. A removable media 1011 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. is attached to the drive 1010 as necessary, so that a computer program read therefrom is installed in the storage part 1008 as necessary.

[0140] In particular, according to embodiments of the present disclosure, the processes described below with reference to the flowcharts can be implemented as a computer software program. For example, embodiments of the present disclosure include a computer program product including a computer program carried on a computer-readable medium, the computer program containing program codes for executing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network by the communication part 1009, and / or installed from the removable media 1011. When the computer program is executed by the central processing unit (CPU) 1001, various functions defined in the methods and apparatuses of the present application are performed. In some embodiments, the electronic device can also include an AI (Artificial Intelligence) processor for processing computing operations related to machine learning.

[0141] It should be noted that the computer-readable medium shown in the disclosure can be a computer-readable signal medium or a computer-readable storage medium or any combination thereof. The computer-readable storage medium may, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any combination thereof. More specific examples of the computer-readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the disclosure, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device or apparatus. In the disclosure, the computer-readable signal medium can include a data signal carried in a baseband or as a part of a carrier wave, which carries computer-readable program code. Such a propagated data signal can take various forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable medium other than the computer-readable storage medium, which can send, propagate or transmit a program for use by or in conjunction with an instruction execution system, device or apparatus. The program code contained on the computer-readable medium can be transmitted by any suitable medium, including but not limited to wireless, wire, optical cable, RF, etc., or any suitable combination thereof.

[0142] The flowcharts and block diagrams in the drawings illustrate the possible implementation architectures, functions and operations of the systems, methods and computer program products according to various embodiments of the disclosure. In this regard, each block in the flowcharts or block diagrams can represent a module, a program segment, or a part of code containing one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur in different order than that shown in the drawings. For example, two blocks that are shown in succession can actually be executed substantially in parallel, and they can also be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams or flowcharts, and the combination of blocks in the block diagrams or flowcharts, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0143] The units described in the embodiments of the present disclosure can be implemented by software, or by hardware, or by a combination of software and hardware. The units described can be located in a single processor, or distributed on multiple processors.

[0144] As another aspect, the present disclosure also provides a computer readable medium, which can be included in the electronic device described in the above embodiments, or can exist independently without being assembled into the electronic device.

[0145] The computer readable medium described above carries one or more programs, which, when executed by the electronic device, cause the electronic device to implement the methods described in the above embodiments. For example, the electronic device can implement each step of the above method embodiments and the like.

[0146] The integrated units in the above embodiments, if realized in the form of software function units and sold or used as independent products, can be stored in the computer readable storage medium described above. Based on such understanding, the technical solutions of the present disclosure, essentially or in part, or all or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes several instructions for causing one or more computer devices (which can be personal computers, servers, or network devices, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present disclosure.

[0147] In the above embodiments of the present disclosure, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0148] In the several embodiments provided by the present disclosure, it should be understood that the terminal device disclosed can be implemented in other ways. Of course, the unit described as the division in the above device embodiment is only a logical division, and there can be another division manner in actual implementation, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interface, unit or module, and can be electrical or other forms.

[0149] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., they can be located in one place, or can be distributed on multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the present embodiment.

[0150] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0151] The above is only the preferred embodiment of the present application, and it should be pointed out that, for those skilled in the art, without departing from the principles of the present application, a number of improvements and refinements can be made, which should be considered as the protection scope of the present application.

Claims

1. A method for identifying gas pipeline leaks, characterized in that, include: Acquire an image of the gas pipeline to be identified; A neural network model, pre-built and trained based on a backbone feature extraction network, a feature fusion network, and a classification prediction network, is used to identify whether there is a gas leak point in the gas pipeline in the image to be identified. The neural network model is constructed using the following methods: a backbone feature extraction network is constructed, which is used for feature extraction; a feature fusion network is constructed by introducing a bidirectional feature pyramid network, which uses cross-scale connections to enhance the feature fusion capability; a small target detection layer is introduced into the classification prediction network to construct the classification prediction network, which is used to identify gas leak points smaller than a preset size; and the neural network model is constructed based on the backbone feature extraction network, the feature fusion network, and the classification prediction network. The construction of the backbone feature extraction network includes: constructing a focusing module, wherein the focusing module is used to slice the input image, transforming the input image into a feature map of a first fixed size, and transforming the first fixed-size feature map into a feature map of a second fixed size through convolution operations of convolution kernels; constructing a CBL module using ordinary convolution Conv, normalized BN, and activation functions; constructing a Res module using two CBL modules as the backbone and residual edges; constructing a CSP module by passing the CBL module, the Res module, and the ordinary convolution Conv as the backbone and the ordinary convolution Conv through BN, activation layers, and the CBL module, wherein the CSP module includes a first CSP module, a second CSP module, a third CSP module, and a fourth CSP module; constructing an SPP module based on the CBL module and a max pooling layer; and constructing the backbone feature extraction network based on the focusing module, the CBL module, the CSP module, and the SPP module. The feature fusion network is constructed by introducing a bidirectional feature pyramid network, including: a preliminary feature layer f4 derived from the fourth CSP module of the feature fusion network is passed through the first CBL module of the feature fusion network to obtain a first enhanced feature layer P1; P1 is upsampled and fused with the preliminary feature layer f3 derived from the third CSP module of the backbone feature extraction network, and the fused feature layer is then passed through the fifth CSP module and the second CBL module of the backbone feature extraction network to obtain a second enhanced feature layer P2; P2 is upsampled and connected with the second preliminary feature layer f2 derived from the second CSP module of the backbone feature extraction network, and the connection is then... The third enhanced feature layer P3 is obtained after passing through the sixth CSP module and a convolutional layer. P3 is then upsampled and connected to the first preliminary feature layer f1 derived from the first CSP module of the backbone feature extraction network. After connection, it passes through the seventh CSP module to obtain the fourth enhanced feature layer P4. P4 passes through the fourth CBL module and is connected to P3 and f2. After passing through the eighth CSP module, it obtains the fifth enhanced feature layer P5. P5 passes through the fifth CBL module and is connected to P2 and f3. After passing through the ninth CSP module, it obtains the sixth enhanced feature layer P6. P6 passes through the sixth CBL module and is connected to P1. After passing through the tenth CSP module, it obtains the seventh enhanced feature layer P7.

2. The method according to claim 1, characterized in that, The neural network model was trained using the following method: Capture images of leaks in buried gas pipelines in urban areas; The acquired images are normalized to a uniform size by scaling them using trilinear interpolation; The normalized images are subjected to data augmentation processing to expand the size of the dataset; The leak points in the dataset are marked to create labels; The dataset is divided into a training set and a test set. The neural network model is trained using the training set and tested using the test set.

3. The method according to claim 2, characterized in that, After normalization, the method further includes performing at least one of the following transformations on the normalized image: random rotation, translation, cropping, scaling, hue, saturation, and brightness.

4. The method according to any one of claims 1 to 3, characterized in that, The gas pipeline is made of PE pipe; the gas pipeline is DN100 pipe.

5. The method according to claim 2, characterized in that, The method further includes: inputting the fourth to seventh enhanced feature layers obtained after feature fusion into four different head classification prediction networks for classification prediction, so as to identify micro, small, medium and large leak points in the gas pipeline image to be identified.

6. A device for identifying gas pipeline leaks, characterized in that, include: The acquisition module is configured to acquire images of the gas pipeline to be identified; The identification module is configured to use a neural network model pre-built and trained based on a backbone feature extraction network, a feature fusion network, and a classification prediction network to identify whether there is a gas leak point in the gas pipeline in the image of the gas pipeline to be identified. The neural network model is constructed using the following methods: a backbone feature extraction network is constructed, which is used for feature extraction; a feature fusion network is constructed by introducing a bidirectional feature pyramid network, which uses cross-scale connections to enhance the feature fusion capability; a small target detection layer is introduced into the classification prediction network to construct the classification prediction network, which is used to identify gas leak points smaller than a preset size; and the neural network model is constructed based on the backbone feature extraction network, the feature fusion network, and the classification prediction network. The construction of the backbone feature extraction network includes: constructing a focusing module, wherein the focusing module is used to slice the input image, transforming the input image into a feature map of a first fixed size, and transforming the first fixed-size feature map into a feature map of a second fixed size through convolution operations of convolution kernels; constructing a CBL module using ordinary convolution Conv, normalized BN, and activation functions; constructing a Res module using two CBL modules as the backbone and residual edges; constructing a CSP module by passing the CBL module, the Res module, and the ordinary convolution Conv as the backbone and the ordinary convolution Conv through BN, activation layers, and the CBL module, wherein the CSP module includes a first CSP module, a second CSP module, a third CSP module, and a fourth CSP module; constructing an SPP module based on the CBL module and a max pooling layer; and constructing the backbone feature extraction network based on the focusing module, the CBL module, the CSP module, and the SPP module. The feature fusion network is constructed by introducing a bidirectional feature pyramid network, including: a preliminary feature layer f4 derived from the fourth CSP module of the feature fusion network is passed through the first CBL module of the feature fusion network to obtain a first enhanced feature layer P1; P1 is upsampled and fused with the preliminary feature layer f3 derived from the third CSP module of the backbone feature extraction network, and the fused feature layer is then passed through the fifth CSP module and the second CBL module of the backbone feature extraction network to obtain a second enhanced feature layer P2; P2 is upsampled and connected with the second preliminary feature layer f2 derived from the second CSP module of the backbone feature extraction network, and the connection is then... The third enhanced feature layer P3 is obtained after passing through the sixth CSP module and a convolutional layer. P3 is then upsampled and connected to the first preliminary feature layer f1 derived from the first CSP module of the backbone feature extraction network. After connection, it passes through the seventh CSP module to obtain the fourth enhanced feature layer P4. P4 passes through the fourth CBL module and is connected to P3 and f2. After passing through the eighth CSP module, it obtains the fifth enhanced feature layer P5. P5 passes through the fifth CBL module and is connected to P2 and f3. After passing through the ninth CSP module, it obtains the sixth enhanced feature layer P6. P6 passes through the sixth CBL module and is connected to P1. After passing through the tenth CSP module, it obtains the seventh enhanced feature layer P7.

7. A system for identifying gas pipeline leaks, characterized in that, include: Gas pipelines; An image acquisition device is configured to acquire images of the gas pipeline via a camera device, as the gas pipeline image to be identified; The apparatus for identifying gas pipeline leaks as described in claim 6 is configured to identify whether a gas leak point exists in the gas pipeline in the image of the gas pipeline to be identified.

Images