A method, device and electronic equipment for detecting damage to ground markings on long-distance pipelines

By building a damage detection model for ground identification of long-distance pipelines, using image processing and deep learning technology, the problems of traditional manual inspections are solved, and high-precision and automated damage detection are achieved, and real-time monitoring and maintenance are supported. .

CN117214189BActive Publication Date: 2024-11-29BEIJING BAOLONG HONGRUI TECH CO LTD +1
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202311486275.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-11-09
Publication Date
2024-11-29
Estimated Expiration
2043-11-09

AI Technical Summary

Technical Problem

The traditional manual inspection method is inefficient in detecting damage to the ground mark of long-distance pipelines, is susceptible to weather and geographical environment, and has problems of misjudgment and misreport, making it difficult to detect damage in a timely manner.

Method used

Build a damage detection model for ground identification of long-distance pipelines, and through image acquisition, preprocessing and model training, use backbone networks, multi-scale feature fusion networks and detection heads to perform damage detection to achieve automated detection.

Benefits of technology

It improves detection accuracy, reduces false alarms and missed reports, reduces labor demand, realizes real-time monitoring and timely repairs, and reduces maintenance costs.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117214189B_ABST
    Figure CN117214189B_ABST
Patent Text Reader

Abstract

The present disclosure discloses a method, device and electronic device for detecting damage to ground markings of long-distance pipelines, the method comprising the following steps: S100: collecting images of ground markings of long-distance pipelines to be tested; S200: preprocessing the images to obtain preprocessed images; S300: constructing and training a model for detecting damage to ground markings of long-distance pipelines; S400: inputting the preprocessed images into the trained model to detect whether the ground markings of long-distance pipelines in the images are damaged. The present disclosure can provide higher detection accuracy by constructing a model for detecting damage to ground markings of long-distance pipelines, thereby reducing false positives and false negatives, and can also reduce the demand for manual labor, thereby improving detection efficiency.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of intelligent recognition and image processing technology, and specifically relates to a method, device and electronic equipment for detecting damage to ground markings on long-distance pipelines. Background Art

[0002] There are inspection needs in many fields for construction area inspection, such as detecting whether there are ground markings for long-distance pipelines, such as inspections in construction sites, fields and other areas where construction is taking place. With the popularization of long-distance pipelines, the safety issues of long-distance pipelines are becoming more and more noticeable. Among them, damage to ground markings on long-distance pipelines is one of the common safety hazards of long-distance pipelines. At the construction site, if the excavation vehicles fail to notice the ground markings on long-distance pipelines during excavation work, the safe operation of the pipelines will be seriously endangered. The traditional inspection method based on manual inspection has low operating efficiency and is subject to subjective assumptions and misjudgments, difficulty in long-term monitoring by manpower, and waste of human resources. It is also easily affected by weather and geographical environment. Summary of the invention

[0003] In view of the deficiencies in the prior art, the purpose of the present invention is to provide a method for detecting damage to ground markings on long-distance pipelines. By constructing a long-distance pipeline ground marking damage detection model, the present invention can provide higher detection accuracy, thereby reducing false alarms and missed alarms, and at the same time can reduce the demand for manual labor, thereby improving detection efficiency.

[0004] To achieve the above objectives, the present disclosure provides the following technical solutions:

[0005] A method for detecting damage to ground markings on long-distance pipelines comprises the following steps:

[0006] S100: collecting ground identification images of the long-distance pipeline to be tested;

[0007] S200: preprocessing the image to obtain a preprocessed image;

[0008] S300: Build and train a long-distance pipeline ground marking damage detection model;

[0009] S400: Input the preprocessed image into the trained model to detect whether the ground marking of the long-distance pipeline in the image is damaged.

[0010] Preferably, in step S200, the preprocessing of the image includes the following steps: adjusting the size of the image and adjusting the color of the image.

[0011] Preferably, in step S300, the long-distance pipeline ground marking damage detection model is trained by the following steps:

[0012] S201: Acquire multiple images containing damaged ground markings of long-distance pipelines, and annotate the images to obtain an annotated image dataset;

[0013] S202: preprocessing the labeled image dataset to obtain a preprocessed image dataset;

[0014] S203: Divide the preprocessed image data set into a training set and a test set;

[0015] S204: Setting training parameters, using the training set to train the model, and iterating the model training through a linear prediction strategy. When the maximum number of iterations is reached, the model training is completed.

[0016] S205: Use the test set to test the trained model. During the test, if the confidence of the model output reaches 0.95, the model test passes; otherwise, adjust the training parameters and retrain the model.

[0017] The present disclosure also provides a long-distance pipeline ground mark damage detection device, the device comprising:

[0018] A collection unit, used to collect ground identification images of the long-distance pipeline to be tested;

[0019] A preprocessing unit, used for preprocessing the image to obtain a preprocessed image;

[0020] A model building and training unit, used to build and train a long-distance pipeline ground marking damage detection model; wherein the model includes a backbone network, a multi-scale feature fusion network and a detection head, the backbone network includes four layers, the first layer includes an image block embedding layer and a FasterNet module, and the second to fourth layers include an image block merging layer and a FasterNet module respectively; a triple attention mechanism is introduced in the multi-scale feature fusion network, so that the model can simultaneously capture the direct relationship between channel attention, spatial attention and weight;

[0021] The detection unit is used to input the preprocessed image into the trained model to detect whether the ground marking of the long-distance pipeline in the image is damaged.

[0022] Preferably, the model building and training unit includes:

[0023] The annotation subunit is used to annotate the acquired multiple images containing the damaged ground markings of the long-distance pipeline to obtain an annotated image data set;

[0024] A preprocessing subunit, used for preprocessing the labeled image data set to obtain a preprocessed image data set;

[0025] A division subunit, used for dividing the preprocessed image data set into a training set and a test set;

[0026] A training subunit, used to train the model using a training set;

[0027] The test subunit is used to test the trained model using the test set.

[0028] The present disclosure also provides a computer storage medium storing computer executable instructions for executing any of the above methods.

[0029] The present disclosure also provides an electronic device, comprising:

[0030] A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein:

[0031] When the processor executes the program, any of the above methods is implemented.

[0032] Compared with the prior art, the present invention has the following beneficial effects:

[0033] 1. Automated inspection: Compared with traditional manual inspection methods, the present invention can automatically collect and process ground identification images and use trained models for damage detection, which can greatly reduce the demand for manual labor and improve inspection efficiency.

[0034] 2. High-precision detection: By building and training a special ground marker damage detection model, it can provide higher detection accuracy, reduce false positives and missed positives, and thus improve the reliability of pipeline ground marker damage detection.

[0035] 3. Real-time monitoring: The present disclosure can promptly detect damage to ground markings, thereby facilitating timely repair measures, reducing maintenance costs and potential pipeline problems.

[0036] 4. Data recording and analysis: By recording and analyzing the collected images and test results, the present disclosure can help establish historical records and trends for predictive maintenance and decision support. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 This is a flow chart of a method for detecting damage to ground markings on long-distance pipelines provided by an embodiment of the present disclosure;

[0038] Figure 2 It is a structural schematic diagram of a long-distance pipeline ground marking damage detection model provided by another embodiment of the present disclosure;

[0039] Figure 3 yes Figure 2Schematic diagram of the structure of the backbone network in the model shown;

[0040] Figure 4 yes Figure 2 Schematic diagram of the structure of the local convolution extraction block (3×3PConv) in the shown model;

[0041] Figure 5 yes Figure 2 Schematic diagram of the structure of the triple attention mechanism (Att) in the shown model;

[0042] Figure 6 yes Figure 2 A schematic diagram of the structure of the detection head in the model shown;

[0043] Figure 7 yes Figure 2 Schematic diagram of the structure of the multi-scale feature fusion network in the shown model. DETAILED DESCRIPTION

[0044] The following will refer to the attached Figures 1 to 7 Specific embodiments of the present disclosure are described in detail. Although specific embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0045] It should be noted that certain words are used in the specification and claims to refer to specific components. Those skilled in the art should understand that technicians may use different nouns to refer to the same component. This specification and claims do not use the difference in nouns as a way to distinguish components, but use the difference in the functions of the components as the criterion for distinction. As mentioned throughout the specification and claims, "including" or "comprising" is an open term, so it should be interpreted as "including but not limited to". The subsequent description of the specification is a preferred embodiment of the present disclosure, but the description is based on the general principles of the specification and is not used to limit the scope of the present disclosure. The scope of protection of the present disclosure shall be determined by the attached claims.

[0046] To facilitate understanding of the embodiments of the present disclosure, further explanation will be given below using specific embodiments as examples in conjunction with the accompanying drawings, and each of the accompanying drawings does not constitute a limitation on the embodiments of the present disclosure.

[0047] In one embodiment, if Figure 1 As shown, the present disclosure provides a method for detecting damage to ground markings on long-distance pipelines, characterized in that the method comprises the following steps:

[0048] S100: collecting ground identification images of the long-distance pipeline to be tested;

[0049] S200: preprocessing the image to obtain a preprocessed image;

[0050] S300: Build and train a long-distance pipeline ground marking damage detection model;

[0051] S400: The preprocessed image is input into the trained model to detect whether the ground markings of the long-distance pipeline, including various signs, signs, color markings, and ground markings, are missing, blurred, tilted, damaged, or faded.

[0052] In another embodiment, in step S200, the preprocessing of the image includes the following steps: adjusting the size of the image to the model input size of 640×640, and adjusting the color of the image to change the overall tone of the image, and further increasing or decreasing the brightness and contrast of the image to change the brightness and darkness of the image.

[0053] In another embodiment, if Figure 2 As shown, the long-distance pipeline ground marking damage detection model includes: a backbone network (Backbone), a multi-scale feature fusion network (Neck) and a detection head (Head).

[0054] In the following, this embodiment respectively introduces in detail the structures and operating principles of the backbone network, the multi-scale feature fusion network and the detection head.

[0055] 1. Backbone network

[0056] like Figure 3 As shown in the figure, the backbone network adopts the FasterNet neural network, which includes four layers. The first layer includes an image block embedding layer Emebedding (which is a regular 4×4 convolution with a step size of 4) and a FasterNet Block. The second to fourth layers include an image block merging layer Merging and a FasterNet Block, respectively. The image block embedding layer Emebedding is used for spatial downsampling and channel number expansion.

[0057] like Figure 3 As shown in the figure, FasterNet Block includes a 3×3PConv layer and two 1×1Conv layers (PConv represents local convolution) connected in sequence. A batch normalization layer BN and an activation function ReLu are set between the two 1×1Conv layers to maintain feature diversity and achieve low latency. In FasterNet Block, the local convolution extraction block 3×3PConv is used as the core layer. Its principle is as follows Figure 4As shown in Figure 1, the 3×3PConv layer only needs to apply a regular convolution extraction block on a portion of the input channels for spatial feature extraction, and keep the remaining channels unchanged. For continuous or regular memory access, the first or last continuous channel is considered as a representative of the entire feature map for calculation. Without loss of generality, the input and output feature maps are considered to have the same number of channels, so the 3×3PConv layer takes only 1 / 16 of the time of a regular convolution.

[0058]

[0059] in, is the height of the image, is the width of the image, is the number of channels of the image, is the convolution kernel size.

[0060] Furthermore, the backbone network also includes an SPPF layer. The main function of the SPPF layer is to capture feature information of different scales. The operation process is as follows:

[0061] First, the SPPF layer divides the input feature map into different sub-regions, then performs a pooling operation on each sub-region, and finally merges the features extracted from sub-regions of different scales to form a feature vector with a fixed dimension, thereby improving the model's ability to detect objects or objects of different sizes.

[0062] Figure 2 In the figure, the SPPF layer is set after the last FasterNet Block of the backbone network. The SPPF layer includes a ConvBNSiLU layer (consisting of a Conv2d layer, a BatchNorm2d layer and a SiLU activation function). The ConvBNSiLU layer connects two branches, where the first branch includes a Concat function and a ConvBNSiLU layer; the second branch includes three sequentially connected two-dimensional maximum pooling layers (MaxPool2d). MaxPool2d performs maximum pooling according to a 2×2 filter and a step size of 2, and selects the maximum value in the window of input data as the output to reduce the size of the feature map and retain important information.

[0063] The backbone network described above achieves much higher operating speeds than other networks on a wide range of devices without compromising the accuracy of various visual tasks, meeting the detection speed requirements of embedded detection modules.

[0064] 2. Multi-scale feature fusion network

[0065] like Figure 7As shown, the multi-scale feature fusion network includes six branches, wherein the first branch includes a Conv layer; the second branch includes an Att layer (Triplet Attention mechanism), a Conv layer, a Fusion layer, a Block layer, a Fusion layer, and a Block layer connected in sequence; the third branch includes an Att layer, a Conv layer, a Fusion layer, a Block layer, and an Upsample layer connected in sequence; the fourth branch includes an Att layer, a Conv layer, and an Upsample layer connected in sequence; the fifth branch includes a Conv layer, a Fusion layer, and a Block layer connected in sequence; the sixth branch has the same structure as the fifth branch, and also includes a Conv layer, a Fusion layer, and a Block layer connected in sequence. At the same time, the Conv layer in the first branch and the Conv layer in the second branch are simultaneously connected to the Fusion layer in the second branch; the second Block layer in the second branch is connected to the Conv layer in the fifth branch; the Conv layer and Block layer in the third branch are simultaneously connected to the Fusion layer in the fifth branch, and the Upsample layer in the third branch is connected to the Fusion layer in the second branch; the Conv layer in the fourth branch is connected to the Fusion layer in the sixth branch, and the Upsample layer in the fourth branch is connected to the Fusion layer in the third branch; the Block layer in the fifth branch is connected to the Conv layer in the sixth branch.

[0066] In the structure of the multi-scale feature fusion network described above, the Att layer is further introduced in this embodiment based on the traditional multi-scale feature fusion network. Its structure is as follows: Figure 5 As shown: Figure 5In the Att layer, the input layer is used to input the features extracted by the backbone network (Input Tensor). After the input layer, three branches are connected. The first branch uses an almost parameter-free attention mechanism to extract channel attention information. On this branch, identity represents the identity operation. The input features extracted by the backbone network (Input Tensor) are Tensor) passes through the Z-Pool module, then through the 7x7 convolution Conv, and finally through the Sigmoid activation function to generate the spatial attention weight. Through element-by-element multiplication, the pixel of each input feature map is multiplied by the spatial attention weight of the corresponding position to generate a weighted output feature map; the second branch is similar to the channel attention branch, but emphasizes capturing spatial attention, the channel C and spatial W dimension interactive capture branch. On this branch, the input feature is first processed by the permute operation, that is, the permute function, and arranged to become H×C×W dimensional features, and then Z-Pool is performed on the H dimension. The subsequent operations are similar to the first branch, and finally it needs to be permuted to become C×H×W dimensional features; the third branch, the channel C and spatial H dimension interactive capture branch, the rest of the operations are the same as the second branch, and will not be repeated. Finally, the output features of the three branches are added to calculate the average Avg.

[0067] The Att layer described above is designed to effectively model channel attention and spatial attention, emphasizing multi-dimensional interactions without reducing the importance of dimensions, eliminating the indirect correspondence between channels and weights, and allowing the model to simultaneously capture the direct relationship between channel attention, spatial attention, and weights. It can effectively improve the expressiveness of the model and enable the model to better understand the multifaceted relationships of the input data.

[0068] Below, this embodiment describes the operating principle of the multi-scale feature fusion network with the above structure as follows:

[0069] The feature map output by the first layer of the backbone network is generated by the convolution operation of the Conv layer in the first branch of the multi-scale feature fusion network (in order to make the features from the bottom and high layers have consistent dimensions and semantics, the feature map will first be transformed, such as using convolution operations) and is denoted as T1; the feature map output by the second layer of the backbone network is generated by the Att attention mechanism and conv convolution operation in the second branch of the multi-scale feature fusion network and is denoted as T2; the feature map output by the third layer of the backbone network is generated by the Att attention mechanism and conv convolution operation in the third branch of the multi-scale feature fusion network and is denoted as T3; the feature map output by the SPPF layer in the backbone network is generated by the Att attention mechanism and conv convolution operation in the fourth branch of the multi-scale feature fusion network and is denoted as T4. After Upsample, T4 is fused with T3 for fusion features (usually using a weighted fusion method to ensure balanced information transmission. The purpose of fusion is to integrate semantic information from different levels to improve performance). The fused feature map is subjected to the Block operation (Block operation is an operation that further processes or transforms the fused features, with the purpose of improving the performance of the network, adapting to the needs of specific tasks, and making better use of these features in subsequent levels.) The generated feature map is denoted as T5. After Upsample, T5 is fused with T2 for fusion features, and then fused with T1 for fusion features through the Block operation. The output feature map is subjected to the Block operation to obtain the input of the detection head Head, which is denoted as T6. T6 is fused with T5 through the conv operation, and then subjected to the Block operation to obtain the input of the detection head Head, which is denoted as T7. After the conv operation, T7 is fused with the feature map of T1 through the Fusion feature, and then subjected to the Block operation to obtain the input of the detection head Head. By fusing the up- and down-sampled feature maps layer by layer, features of different scales can be better integrated and utilized, effectively solving the problems of information loss and redundancy in traditional feature pyramid networks when extracting features of different scales. It has strong robustness when dealing with complex scenes such as scale changes and occlusions.

[0070] 3. Detection head

[0071] The detection head in this model is a dynamic detection head, and its structure is as follows: Figure 6 shown. Figure 6 In the detection head, there are three sequential attention mechanisms, namely, the scale-aware attention mechanism π L , spatial perception attention mechanism π s and task-aware attention mechanism π c , where the scale-aware attention mechanism π LFrom bottom to top, it includes the average pooling layer (Avg pool), 1x1 convolution layer conv, ReLU activation function and Hard Sigmoid activation function connected in sequence. The input features are multiplied with the input features after the above operations to complete scale perception. Spatial perception attention mechanism π s Scale-aware attention π L The output of is taken as input, which includes the index operation index, 3x3 convolution conv, position offset calculation offset and Sigmoid function connected in sequence from bottom to top. First, the spatial perception attention mechanism π s The input features are indexed and convolved 3x3 times, and then divided into two branches. One branch is used to calculate the position offset, and the other branch calculates the output of the Sigmoid function. Finally, the two branches and the input are multiplied element by element, and the output is used as the task-aware attention π. c Input. Task-aware attention mechanism π c From bottom to top, it includes the average pooling layer (Avg pool), the fully connected layer FC, the activation function relu, the fully connected layer FC, the normalization layer and the vector [1,0,0,0]. First, the task-aware attention mechanism π c Spatial Aware Attention Mechanism π s The output of is average pooled, then linearly transformed through the fully connected layer fc, then through the ReLU activation function, and then through the fully connected layer again. Finally, the final features are normalized by normalize, and the normalized features are multiplied with the given vector [1,0,0,0]. Finally, the result of the multiplication with the vector is multiplied element by element with the input feature to obtain the final task-aware attention mechanism π c Output.

[0072] For the detection head, a tensor is given as F, and W is the output feature map.

[0073]

[0074] in, , , They represent attention in three dimensions: L (scale), S (space), and C (task).

[0075] Scale-aware attention mechanism π L It is performed on the level dimension. Feature maps of different levels correspond to different target scales. Increasing attention at the level can enhance the scale perception ability of target detection. The calculation formula is as follows:

[0076]

[0077] Among them, f(·) is a linear function, approximated by 1*1 convolution, and α is a Hard-sigmoid function.

[0078] Spatial Aware Attention Mechanism π s Executed in the spatial dimension, different spatial positions correspond to the geometric transformation of the target. Increasing attention in the spatial dimension can enhance the spatial position perception ability of the target detector. The calculation formula is as follows:

[0079]

[0080] in, represents different scale features, K represents the number of sparse sampling positions, k represents the number of different sparse sampling positions, L represents the scale, and C represents the task dimension. Feature maps of different scales representing different numbers of sparse sampling locations, + Indicates the offset of the self-learning space The position obtained after the position is obtained, in order to focus on the discriminant area, Indicates location The self-learned importance scalar at , both of which are learned from the input features F from the middle layer.

[0081] Task-aware attention mechanism π c Executed on the channel dimension, different channels correspond to different tasks. Adding attention on the input dimension can enhance the object detection's perception of different tasks.

[0082]

[0083] in, represents the feature slice of the c-th channel, [ , , , ] T =θ represents the hyperfunction for learning the activation threshold. θ first performs global average pooling on the L×S dimension, then uses two fully connected layers and a normalization layer, and finally applies the shifted sigmoid function to normalize the output to [-1, 1].

[0084] The dynamic detection head can dynamically adapt and adjust its working method according to different situations and tasks. Specifically, it improves the representation ability of the object detection head by coherently combining multiple self-attention mechanisms between feature levels for scale perception, between spatial positions for spatial perception, and between output channels for task perception, without any computational overhead.

[0085] Below, this embodiment takes a single image in the training set as an example to describe the learning process of the model.

[0086] Input an RGB long-distance pipeline ground marking image of size (640,640), represented as (1,3,640,640), where 1 represents the batch size of samples processed by the model at one time, 3 represents the number of color channels in the image, that is, RGB color image, and (640,640) represents the height and width of the image. The input image is first embedded by the image block embedding layer Emebedding. Emebedding divides the input image into non-overlapping small blocks, and then applies a convolution layer to each block for embedding. In this example, each (4,4) image block will be embedded as a feature vector, and a feature map of size (1,40,160,160) will be obtained. The backbone network consists of multiple layers, each of which consists of a basic module FasterNet Block and an image block merging layer Merging. In each basic module, the feature map will pass through a convolution layer, batch normalization, and activation function. These blocks gradually extract and process features to have different resolutions and semantic information. At the end of each basic module, the resolution of the feature map is halved and the number of channels is increased. From different layers of the backbone network, the following feature maps can be obtained:

[0087] Feature map is interpreted as: [batch size, feature dimension, image width, image height]

[0088] Output of layer 1: feature map size is [1,40,160,160]

[0089] Output of layer 2: feature map size is [1,80,80,80]

[0090] Output of layer 3: feature map size is [1,160,40,40]

[0091] Output of layer 4: feature map size is [1,320,20,20]

[0092] Next, the extracted feature maps are input into the multi-scale feature fusion network for feature fusion. First, the feature map of size (1,320,20,20) is transformed to (1,320,40,40) through deconvolution (upsampling) operation so that it can be fused with the feature map of size (1,160,40,40). Next, the feature map of size (1,160,40,40) is fused with the upsampled feature map of size (1,320,40,40). The fused feature map of size (1,320,40,40) is transformed to (1,320,80,80) through deconvolution (upsampling) operation and fused with the feature map of size (1,80,80,80). The feature map of size (1,80,80,80) and the upsampled feature map of size (1,320,80,80) are fused to generate a fused feature map of size (1,320,80,80). Similarly, the fused feature map of size (1,320,80,80) is transformed into (1,320,160,160) through deconvolution (upsampling) operation and fused with the feature map of size (1,40,160,160). Finally, the feature map of size (1,40,160,160) and the upsampled feature map of size (1,320,160,160) are fused to obtain a fused feature map of size (1,320,160,160), whose dimension is consistent with the input feature map.

[0093] Through upsampling and feature fusion operations, the multi-scale feature fusion network successfully fuses feature maps of different scales, making the fused feature maps (1,160,40,40), (1,80,80,80) and (1,40,160,160) fully contain rich visual information. Subsequently, these fused feature maps containing highly rich visual information are seamlessly passed to the detection head.

[0094] Inside the detection head, the fused feature maps are carefully processed through sophisticated convolution, linear transformation, and dynamic head processing. Inside the detection head, the processing of feature maps undergoes the processing of three key attention mechanisms: scale, space, and task, as follows:

[0095] First, through the scale-aware attention mechanism π L The features are scaled: the average pooling operation avg pool reduces the spatial resolution, the 1x1 convolution conv performs linear transformation, the relu activation enhances the nonlinear expression, the hard sigmoid activation generates scale-aware weights, and finally multiplies the input data element by element to obtain scale-aware features.

[0096] Next, through the spatially aware attention mechanism π s The features are spatially processed: the index operation index selects the position or channel of interest, the 3x3 convolution conv performs local linear transformation to capture spatial information and feature interactions, the position offset calculation offset determines the position offset, the sigmoid weight generation adjusts the attention in space, and the outputs of the last two branches are fused to generate the final feature representation to capture the spatial relationship of the input data.

[0097] Finally, through the task-aware attention mechanism π c Task the features: average pooling avg pool is performed on the spatial perception attention output, linear transformation is performed on the fully connected layer fc, relu activation introduces nonlinear characteristics, fully connected fc adjusts the features again, normalizes the operation to ensure the numerical range, adds the predefined vector [1,0,0,0] to introduce task-related information, and finally adds the input data element by element to generate the output of the task-aware attention mechanism. This comprehensive multi-level feature processing process enables the model to dynamically adjust the feature representation to adapt to different scales, spaces and task requirements, thereby improving performance and adaptability.

[0098] In this embodiment, the dynamic detection head can dynamically adjust the structure and parameters of the head according to the characteristics of different scales and features, so that the model can better adapt to targets of different scales, thereby improving the accuracy and robustness of detection. Through dynamic head processing, targets in the ground identification map of long-distance pipelines can be processed more flexibly, regardless of their size, shape and location.

[0099] While performing target detection, a series of important loss functions are introduced to train and optimize the model. DIOU loss function L DIoU It is used to measure the relationship between the distance and IoU between the predicted box and the true box, so as to more accurately evaluate the accuracy of the prediction. By minimizing the DIOU loss, the model can better predict the location and shape of the target, thereby improving the detection accuracy. The calculation formula is as follows:

[0100]

[0101]

[0102] Among them, IoU represents the intersection-over-union ratio of two bounding boxes A and B. , Represent the center points of frame A and frame B respectively, It represents the calculation of the Euclidean distance between two center points, and d represents the diagonal distance of the smallest rectangle that can cover both the A box and the B box.

[0103] Non-maximum suppression is also an important step, which is used to remove redundant prediction boxes and retain the target box with the highest confidence. By comparing the IoU of different prediction boxes and screening out high-confidence and non-overlapping target boxes, the final detection result is ensured to be more accurate and compact. Specifically, the method includes the following steps:

[0104] 1. Score screening: First, for each candidate box, calculate its prediction score (usually the confidence score of the target). Then, sort all candidate boxes in descending order according to the score.

[0105] 2. Select high-scoring boxes: Start with the candidate box with the highest score, mark it as selected, and then check the IoU of other unmarked boxes with that box. If the IoU is greater than a certain threshold (usually 0.5), the box is marked as suppressed because it overlaps with the selected high-scoring box.

[0106] 3. Suppress overlapping boxes: For boxes marked as suppressed, remove them from the list of candidate boxes. This ensures that only one high-score box is retained for the same target.

[0107] 4. Repeat: Continue to process the next unlabeled box with the highest score, repeating steps 2 and 3 until all candidate boxes are processed.

[0108] As the prediction phase unfolds, it is possible to accurately determine whether there is damage to the ground markings, as well as the specific location and category of each target. Ultimately, these powerful detection results can be effectively transmitted back to the ground station, providing an important basis for subsequent alarm notifications, maintenance and processing.

[0109] The model can identify the damaged images of long-distance pipeline ground signs, learn texture and detail features: damaged ground signs may show features such as fragmentation, cracks, and wear, shape and geometric features: damaged ground signs may cause changes in the shape of the signs, such as cracks causing deformation of the edges of the signs. As well as features such as color and brightness changes and specific signs. Throughout the process, feature fusion and detection head work together to build an efficient and accurate ground sign damage detection system, which brings significant value to applications in real scenarios.

[0110] In another embodiment, in step S300, the long-distance pipeline ground mark damage detection model is trained by the following steps:

[0111] S201: Acquire multiple images containing damaged ground markings of long-distance pipelines, and annotate the images to obtain an annotated image dataset;

[0112] S202: preprocessing the labeled image dataset to obtain a preprocessed image dataset;

[0113] S203: Divide the preprocessed image data set into a training set and a test set;

[0114] S204: Set training parameters, for example, set the initial learning rate to 0.01, the batch size to 8, the total number of training rounds to 300, the initial weight decay to 0.0001, select SGD as the optimizer, train the model using the training set, and iterate the model through the linear prediction strategy (i.e., train the model within a given number of training times). When the maximum number of iterations is reached, the model training is completed;

[0115] S205: Use the test set to test the trained model. During the test, if the confidence of the model output reaches 0.95 or above, the model test passes; otherwise, adjust the training and retrain the model.

[0116] In another embodiment, the present disclosure further provides a long-distance pipeline ground mark damage detection device, comprising:

[0117] A collection unit, used to collect ground identification images of the long-distance pipeline to be tested;

[0118] A preprocessing unit, used for preprocessing the image to obtain a preprocessed image;

[0119] A model building and training unit, used to build and train a long-distance pipeline ground marking damage detection model; wherein the model includes a backbone network, a multi-scale feature fusion network and a detection head, the backbone network includes four layers, the first layer includes an image block embedding layer and a FasterNet module, and the second to fourth layers include an image block merging layer and a FasterNet module respectively; a triple attention mechanism is introduced in the multi-scale feature fusion network, so that the model can simultaneously capture the direct relationship between channel attention, spatial attention and weight;

[0120] The detection unit is used to input the preprocessed image into the trained model to detect whether the ground marking of the long-distance pipeline in the image is damaged.

[0121] In another embodiment, the model building and training unit includes:

[0122] The annotation subunit is used to annotate the acquired multiple images containing the damaged ground markings of the long-distance pipeline to obtain an annotated image data set;

[0123] A preprocessing subunit, used for preprocessing the labeled image data set to obtain a preprocessed image data set;

[0124] A division subunit, used for dividing the preprocessed image data set into a training set and a test set;

[0125] A training subunit, used to train the model using a training set;

[0126] The test subunit is used to test the trained model using the test set.

[0127] In another embodiment, the present disclosure further provides a computer storage medium storing computer executable instructions for executing any of the above methods.

[0128] In another embodiment, the present disclosure further provides an electronic device, including:

[0129] A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein:

[0130] When the processor executes the program, any of the above methods is implemented.

[0131] The applicant of the present disclosure has described the embodiments of the present disclosure in detail in conjunction with the drawings of the specification, but those skilled in the art should understand that the above embodiments are only preferred implementation examples of the present disclosure and are not limited to the above specific embodiments. The detailed description is intended to help readers better understand the spirit of the present disclosure, but is not intended to limit the scope of protection of the present disclosure. On the contrary, any improvements or changes based on the inventive spirit of the present disclosure should be included in the scope of protection of the present disclosure.

Claims

1. A method for detecting damage to ground markings on long-distance pipelines, characterized in that: The method comprises the following steps: S100: collecting ground identification images of the long-distance pipeline to be tested; S200: preprocessing the image to obtain a preprocessed image; S300: Build and train a long-distance pipeline ground marking damage detection model; The model includes a backbone network, a multi-scale feature fusion network and a detection head. The backbone network adopts a FasterNet neural network, which includes four layers. The first layer includes an image block embedding layer and a FasterNet module, and the second to fourth layers include an image block merging layer and a FasterNet module respectively; the FasterNet module includes a 3×3PConv layer and two 1×1Conv layers connected in sequence, and a batch normalization layer BN and an activation function ReLu are arranged between the two 1×1Conv layers; the backbone network also includes an SPPF layer, which is arranged after the last FasterNet module of the backbone network, and the SPPF layer includes a ConvBNSiLU layer, and the ConvBNSiLU layer connects two branches, wherein the first branch includes a Concat function and a ConvBNSiLU layer; the second branch includes three two-dimensional maximum pooling layers connected in sequence; A triple attention mechanism is introduced into the multi-scale feature fusion network, which is used for the model to simultaneously capture the direct relationship between channel attention, spatial attention and weights; The multi-scale feature fusion network includes six branches, wherein the first branch includes a Conv layer; the second branch includes an Att layer, a Conv layer, a Fusion layer, a Block layer, a Fusion layer and a Block layer connected in sequence; the third branch includes an Att layer, a Conv layer, a Fusion layer, a Block layer and an Upsample layer connected in sequence; the fourth branch includes an Att layer, a Conv layer and an Upsample layer connected in sequence; the fifth branch includes a Conv layer, a Fusion layer and a Block layer connected in sequence; the sixth branch has the same structure as the fifth branch and also includes a Conv layer, a Fusion layer and a Block layer connected in sequence; and at the same time, the Conv layer in the first branch and the Conv layer in the second branch are simultaneously connected to the Fusion layer in the second branch; the second Block layer in the second branch is connected to the Conv layer in the fifth branch; the Conv layer and the Block layer in the third branch are simultaneously connected to the Fusion layer in the fifth branch, and the Upsample layer in the third branch is connected to the Fusion layer in the second branch; the Conv layer in the fourth branch is connected to the Fusion layer in the sixth branch, and the fourth branch The Upsample layer in the network is connected to the Fusion layer in the third branch; the Block layer in the fifth branch is connected to the Conv layer in the sixth branch; the Att layer includes an input layer for inputting the features extracted by the backbone network. The input layer is connected to three branches. The first branch uses an attention mechanism to extract channel attention information. In this branch, identity represents an identity operation. The features extracted by the backbone network are input through the Z-Pool module, then through the 7x7 convolution Conv, and finally through the Sigmoid activation function to generate spatial attention weights. Element multiplication, multiplying the pixel of each input feature map by the spatial attention weight of the corresponding position to generate a weighted output feature map; the second branch emphasizes capturing spatial attention, channel C and spatial W dimension interactive capture branch, on which the input features are first arranged by the permute function to become H×C×W dimensional features, then Z-Pool is performed on the H dimension, and finally permuter is required to become C×H×W dimensional features; the third branch, channel C and spatial H dimension interactive capture branch, the rest of the operations are the same as the second branch, and finally the output features of the three branches are added to calculate Avg; The detection head is a dynamic detection head, which includes three sequential attention mechanisms, namely, scale-aware attention mechanism π L , spatial perception attention mechanism π s and task-aware attention mechanism π c , where the scale-aware attention mechanism π L From bottom to top, it includes the average pooling layer, 1×1 convolution layer conv, ReLU activation function and Hard Sigmoid activation function connected in sequence. The input features are multiplied with the input features after the above operations to complete scale perception; spatial perception attention mechanism π s Perceiving attention at scale π L The output of is taken as input, which includes the index operation index, 3×3 convolution conv, position offset calculation offset and Sigmoid function connected in sequence from bottom to top. First, the spatial perception attention mechanism π s The input features are indexed and convolved 3×3 times, and then divided into two branches. One branch is used to calculate the position offset, and the other branch calculates the output of the Sigmoid function. Finally, the two branches and the input are multiplied element by element, and the output is used as the task-aware attention. π c Input; Task-aware attention mechanism π c From bottom to top, it includes the average pooling layer, the fully connected layer FC, the activation function relu, the fully connected layer FC, the normalization layer and the vector. First, the task-aware attention mechanism π c Attention mechanism for spatial perception π s The output of is average pooled, then linearly transformed through the fully connected layer fc, then through the ReLU activation function, and then through the fully connected layer again. Finally, the final features are normalized by normalize, and the normalized features are multiplied with the given vector. Finally, the result of the vector multiplication is multiplied element by element with the input feature to obtain the final task-aware attention mechanism. π c Output; S400: input the preprocessed image into the trained model to detect whether the ground marking of the long-distance pipeline in the image is damaged.

2. The method according to claim 1, characterized in that In step S200, the preprocessing of the image includes the following steps: adjusting the size of the image and adjusting the color of the image.

3. The method according to claim 1, characterized in that In step S300, the long-distance pipeline ground mark damage detection model is trained by the following steps: S201: Acquire multiple images containing damaged ground markings of long-distance pipelines, and annotate the images to obtain an annotated image dataset; S202: preprocessing the labeled image dataset to obtain a preprocessed image dataset; S203: Divide the preprocessed image data set into a training set and a test set; S204: Setting training parameters, using the training set to train the model, and iterating the model training through a linear prediction strategy. When the maximum number of iterations is reached, the model training is completed. S205: Use the test set to test the trained model. During the test, if the confidence of the model output reaches 0.95, the model test passes; otherwise, adjust the training parameters and retrain the model.

4. A long-distance pipeline ground mark damage detection device, characterized in that: The device comprises: A collection unit, used to collect ground identification images of the long-distance pipeline to be tested; A preprocessing unit, used for preprocessing the image to obtain a preprocessed image; A model building and training unit is used to build and train a long-distance pipeline ground marking damage detection model; wherein, the model includes a backbone network, a multi-scale feature fusion network and a detection head, the backbone network adopts a FasterNet neural network, the network includes four layers, the first layer includes an image block embedding layer and a FasterNet module, the second to fourth layers include an image block merging layer and a FasterNet module respectively; the FasterNet module includes a 3×3PConv layer and two 1×1Conv layers connected in sequence, and a batch normalization layer BN and an activation function ReLu are arranged between the two 1×1Conv layers; the backbone network also includes an SPPF layer, and the SPPF layer is arranged at the last FasterNet layer of the backbone network After the module, the SPPF layer includes a ConvBNSiLU layer, which connects two branches, wherein the first branch includes a Concat function and a ConvBNSiLU layer; the second branch includes three two-dimensional maximum pooling layers connected in sequence; a triple attention mechanism is introduced into the multi-scale feature fusion network, which is used for the model to simultaneously capture the direct relationship between channel attention, spatial attention and weights; the multi-scale feature fusion network includes six branches, wherein the first branch includes a Conv layer; the second branch includes an Att layer, a Conv layer, a Fusion layer, a Block layer, a Fusion layer and a Block layer connected in sequence; the third branch includes an Att layer, a Conv layer, a Fusion layer, a Block layer, and an Upsample layer connected in sequence; the fourth branch includes an Att layer, a Conv layer and Upsample layer; the fifth branch includes a Conv layer, a Fusion layer and a Block layer connected in sequence; the sixth branch has the same structure as the fifth branch, and also includes a Conv layer, a Fusion layer and a Block layer connected in sequence; and at the same time, the Conv layer in the first branch and the Conv layer in the second branch are simultaneously connected to the Fusion layer in the second branch; the second Block layer in the second branch is connected to the Conv layer in the fifth branch; the Conv layer and the Block layer in the third branch are simultaneously connected to the Fusion layer in the fifth branch, and the Upsample layer in the third branch is connected to the Fusion layer in the second branch; the Conv layer in the fourth branch is connected to the Fusion layer in the sixth branch, and the Upsample layer in the fourth branch is connected to the Fusion layer in the third branch; the Block layer in the fifth branch is connected to the Conv layer in the sixth branch;Among them, the Att layer includes an input layer for inputting features extracted by the backbone network. Three branches are connected after the input layer. Among them, the first branch adopts an attention mechanism to extract channel attention information. On this branch, identity represents an identity operation. The input features extracted by the backbone network pass through the Z-Pool module, then pass through the 7x7 convolution Conv, and finally pass through the Sigmoid activation function to generate spatial attention weights. Through element-by-element multiplication, the pixels of each input feature map are multiplied by the spatial attention weights of the corresponding positions to generate a weighted output feature map; the second branch emphasizes capturing spatial attention, the channel C and the spatial W dimension interactive capture branch, on which the input features are first arranged by the permute function to become H×C×W dimensional features, then Z-Pool is performed on the H dimension, and finally permuter is required to become C×H×W dimensional features; the third branch, the channel C and the spatial H dimension interactive capture branch, the rest of the operations are the same as the second branch, and finally the output features of the three branches are added to obtain Avg; the detection head is a dynamic detection head, and the detection head includes three serialized attention mechanisms, namely, scale-aware attention mechanisms; π L , spatial perception attention mechanism π s and task-aware attention mechanism π c , where the scale-aware attention mechanism π L From bottom to top, it includes the average pooling layer, 1×1 convolution layer conv, ReLU activation function and Hard Sigmoid activation function connected in sequence. The input features are multiplied with the input features after the above operations to complete scale perception; spatial perception attention mechanism π s Perceiving attention at scale π L The output of is taken as input, which includes the index operation index, 3×3 convolution conv, position offset calculation offset and Sigmoid function connected in sequence from bottom to top. First, the spatial perception attention mechanism π s The input features are indexed and convolved 3×3 times, and then divided into two branches. One branch is used to calculate the position offset, and the other branch calculates the output of the Sigmoid function. Finally, the two branches and the input are multiplied element by element, and the output is used as the task-aware attention. π c Input; Task-aware attention mechanism π c From bottom to top, it includes the average pooling layer, the fully connected layer FC, the activation function relu, the fully connected layer FC, the normalization layer and the vector. First, the task-aware attention mechanism π c Attention mechanism for spatial perception π s The output of is average pooled, then linearly transformed through the fully connected layer fc, then through the ReLU activation function, and then through the fully connected layer again. Finally, the final features are normalized by normalize, and the normalized features are multiplied with the given vector. Finally, the result of the vector multiplication is multiplied element by element with the input feature to obtain the final task-aware attention mechanism. π c Output: The detection unit is used to input the preprocessed image into the trained model to detect whether the ground marking of the long-distance pipeline in the image is damaged.

5. The device according to claim 4, characterized in that The model building and training unit includes: The annotation subunit is used to annotate the acquired multiple images containing the damaged ground markings of the long-distance pipeline to obtain an annotated image data set; A preprocessing subunit, used for preprocessing the labeled image data set to obtain a preprocessed image data set; A division subunit, used for dividing the preprocessed image data set into a training set and a test set; A training subunit, used to train the model using a training set; The test subunit is used to test the trained model using the test set.

6. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method according to any one of claims 1 to 3 is implemented.

Citation Information

Patent Citations

  • Low-illumination target detection method based on NDF-Net

    CN114037938A

  • Multi-scale target detection method based on attention mechanism

    CN115063573A