Outdoor pipeline corrosion detection method based on visual identification of unmanned aerial vehicle and related device

Abstract

The embodiment of the invention provides an outdoor pipeline corrosion detection method based on visual identification of an unmanned aerial vehicle and a related device. The method comprises the following steps: acquiring an outdoor pipeline corrosion image; the outdoor pipeline corrosion image is input to a target detection model for corrosion detection, an outdoor pipeline corrosion detection result is obtained, the target detection model comprises a backbone network, a neck network and a detection head, the backbone network is used for carrying out feature extraction on the outdoor pipeline corrosion image to obtain multi-scale features, and the neck network is used for carrying out feature extraction on the multi-scale features; the neck network is used for carrying out feature fusion on the multi-scale features to obtain fusion features, the detection head is used for carrying out classification and rotating frame regression prediction on the fusion features to obtain an outdoor pipeline corrosion detection result, and the outdoor pipeline corrosion detection result comprises corrosion area positioning information and a detection frame with confidence; and generating a detection report according to the outdoor pipeline corrosion detection result, and sending the detection report to the user terminal. On the basis, the outdoor pipeline corrosion detection precision and efficiency can be improved.

Description

Technical Field

[0001] This invention relates to the field of UAV visual inspection technology, and in particular to an outdoor pipeline corrosion detection method and related apparatus based on UAV visual recognition. Background Technology

[0002] Factory outdoor pipelines are widely distributed in complex environments such as elevated, winding, and inclined sections. Their long-term exposure to the outdoor environment makes them susceptible to corrosion from wind and rain, leading to rust and potentially causing safety risks such as leaks and explosions. For example, oil pipelines crossing river valleys or steep slopes are often erected using bridge supports or cables, making them vulnerable to wind, rain, and sun exposure, resulting in corrosion. Engineering pipelines (such as water and natural gas pipelines) crossing urban obstacle zones (such as railways and highways) may use culverts, which are also prone to corrosion over time. Currently, common corrosion detection methods include manual inspection, laser scanning, and infrared thermal imaging. These methods suffer from problems such as long inspection cycles, limited accuracy, inconvenient observation angles, and high costs. Especially for irregular structures such as bends and inclined pipes, traditional detection models cannot accurately reconstruct their true position and angle, easily leading to missed detections or misjudgments. Therefore, improving the accuracy and efficiency of outdoor pipeline corrosion detection has become an urgent technical problem to be solved. Summary of the Invention

[0003] This invention provides a method and related device for detecting corrosion of outdoor pipelines based on UAV visual recognition, which can comprehensively improve the accuracy and efficiency of outdoor pipeline corrosion detection.

[0004] In a first aspect, embodiments of the present invention provide a method for detecting corrosion of outdoor pipelines based on UAV visual recognition, comprising:

[0005] Images of outdoor pipe corrosion are obtained by taking pictures of the factory's outdoor pipes using a drone.

[0006] The outdoor pipeline corrosion image is input into a pre-trained target detection model for corrosion detection to obtain the outdoor pipeline corrosion detection result. The target detection model includes a backbone network, a neck network, and a detection head. The backbone network is used to extract features from the outdoor pipeline corrosion image to obtain multi-scale features. The neck network is used to fuse the multi-scale features to obtain fused features. The detection head is used to classify the fused features and perform bounding box regression prediction to obtain the outdoor pipeline corrosion detection result. The outdoor pipeline corrosion detection result includes corrosion area location information and detection boxes with confidence scores.

[0007] An inspection report is generated based on the outdoor pipeline corrosion detection results, and the inspection report is sent to the user terminal for visual display.

[0008] In some embodiments, the training method of the object detection model includes:

[0009] Batch acquisition of publicly available, free images of pipe corrosion from the internet using web crawlers;

[0010] The images of internet pipeline corrosion are filtered to obtain internet pipeline corrosion image data;

[0011] Drones were used to take aerial photos of the factory's outdoor pipelines at regular intervals to obtain real-world images of pipeline corrosion.

[0012] Based on the internet pipeline corrosion image data and the real-world pipeline corrosion image data, an image dataset is obtained;

[0013] The image dataset is preprocessed to obtain the target image dataset;

[0014] The target image dataset is annotated with a rotating bounding box using a preset parameter format to obtain a rotating bounding box corrosion detection dataset.

[0015] The target detection model is trained based on the rotating frame corrosion detection dataset to obtain the trained target detection model.

[0016] In some embodiments, the image dataset is preprocessed to obtain a target image dataset, including:

[0017] All images in the image dataset are cropped according to a preset size to obtain the first image dataset;

[0018] The first image dataset is subjected to geometric transformation enhancement to obtain the second image dataset, wherein the geometric transformation enhancement includes random rotation, scaling, translation and mirror flipping;

[0019] The second image dataset is subjected to illumination enhancement to obtain a third image dataset, wherein the illumination enhancement includes gamma correction, random brightness adjustment and contrast variation;

[0020] The third image dataset is subjected to environmental interference enhancement to obtain the target image dataset, wherein the environmental interference enhancement includes adding Gaussian noise, random occlusion, and motion blur.

[0021] In some embodiments, the backbone network includes standard convolution, residual module C3K2, channel and spatial attention mechanism CBAM, spatial pyramid pooling SPPF, and a C2PSA module that integrates attention and multi-scale features. The standard convolution uses a 3*3 kernel and a stride of 2. The residual module C3K2 is constructed from C3K, a bottleneck module, and a CBS module. The CBS module consists of convolutional layers, batch normalization layers, and SiLU activation functions. The spatial pyramid pooling SPPF is used to extract and encode features from the image at different scales and generate fixed-length feature vectors. The C2PSA module that integrates attention and multi-scale features is used to extract multi-scale features using multiple convolutional kernels. The convolution operations of different kernels are performed in parallel. The feature maps are then concatenated, and the feature channels are weighted. Finally, the generated attention weights are applied to each feature map to achieve point-by-point weighting of the channels.

[0022] In some embodiments, the Channel and Spatial Attention Mechanism (CBAM) includes a channel attention module and a spatial attention module. The channel attention module performs global max pooling and global average pooling operations on each channel of the input feature map to obtain two channel description vectors, representing the global maximum feature and average feature of each channel, respectively. These two channel description vectors are then input into a shared fully connected layer to learn the attention weights for each channel. A sigmoid activation function is then used to ensure that the attention weights are between 0 and 1. Finally, the obtained attention weights are multiplied by each channel of the original feature map to obtain the attention-weighted channel feature map. The spatial attention module performs inter-channel max pooling and average pooling on the channel feature map to obtain two feature maps that are spatially identical to the original feature map and have one channel. These two feature maps are then concatenated along the channel dimension and fused into a spatial attention feature map using a 7×7 convolution operation. Finally, a sigmoid activation function is used to restrict the attention weights to between 0 and 1, and the attention weights are multiplied by the original feature map to obtain the spatially attention-weighted spatial feature map.

[0023] In some embodiments, the neck network employs a path aggregation feature pyramid network (PAFPN) structure.

[0024] In some embodiments, the training method for the detection head includes:

[0025] Determine the classification loss function and the corresponding classification weight parameters;

[0026] Determine the loss function and corresponding rotation regression weight parameters for the rotation box regression;

[0027] The detection head is trained using the classification loss function, the classification weight parameters, the loss function of the rotated bounding box regression, and the rotation regression weight parameters to obtain the trained detection head.

[0028] Secondly, embodiments of the present invention also provide an outdoor pipeline corrosion detection device based on UAV visual recognition, the device comprising:

[0029] The image acquisition module is used to acquire images of outdoor pipeline corrosion, wherein the images of outdoor pipeline corrosion are obtained by drones taking pictures of the factory's outdoor pipelines during patrols.

[0030] The detection module is used to input the outdoor pipeline corrosion image into a pre-trained target detection model for corrosion detection, and obtain the outdoor pipeline corrosion detection result. The target detection model includes a backbone network, a neck network, and a detection head. The backbone network is used to extract features from the outdoor pipeline corrosion image to obtain multi-scale features. The neck network is used to fuse the multi-scale features to obtain fused features. The detection head is used to classify the fused features and perform bounding box regression prediction to obtain the outdoor pipeline corrosion detection result. The outdoor pipeline corrosion detection result includes corrosion area location information and detection boxes with confidence scores.

[0031] The output module is used to generate an inspection report based on the outdoor pipeline corrosion detection results and send the inspection report to the user terminal for visualization.

[0032] Thirdly, embodiments of the present invention also provide an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, when the processor executes the computer program, it implements the outdoor pipeline corrosion detection method based on UAV visual recognition as described in the first aspect.

[0033] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing computer-executable instructions for performing the outdoor pipeline corrosion detection method based on UAV visual recognition as described in the first aspect.

[0034] The present invention provides a method and related apparatus for detecting corrosion of outdoor pipelines based on UAV visual recognition. The method includes: capturing images of corrosion on outdoor pipelines obtained by UAV patrol and photography of factory outdoor pipelines; inputting these images into a target detection model with a rotating bounding box for corrosion detection; the target detection model employing an integrated design of backbone network, neck network, and detection head; the backbone network extracting features from the corrosion images to obtain multi-scale features; the neck network fusing the multi-scale features to obtain fused features, improving the multi-scale feature fusion capability; and the detection head using joint optimization of rotating bounding box regression and classification to classify and predict the fused features, outputting outdoor pipeline corrosion detection results including corrosion area location information and detection boxes with confidence scores, significantly improving the recognition accuracy of bent, tilted, or irregular corrosion areas. Based on the outdoor pipeline corrosion detection results, a detection report visually displaying the pipeline corrosion distribution is generated and sent to a user terminal for visualization, facilitating quick understanding of the detection results. The entire detection process is efficient, fast, and highly accurate. Based on this, the embodiments of the present invention can comprehensively improve the accuracy and efficiency of outdoor pipeline corrosion detection. Attached Figure Description

[0035] Figure 1A This is a flowchart of an outdoor pipeline corrosion detection method based on UAV visual recognition provided in one embodiment of the present invention;

[0036] Figure 1B This is an overall structural diagram of a target detection model provided in one embodiment of the present invention;

[0037] Figure 2 This is a schematic diagram of the residual module C3K2 provided in one embodiment of the present invention;

[0038] Figure 3 This is a schematic diagram of the channel and spatial attention mechanism (CBAM) provided in one embodiment of the present invention;

[0039] Figure 4 This is a schematic diagram of the structure of a spatial pyramid pooled SPPF provided in one embodiment of the present invention;

[0040] Figure 5 This is a schematic diagram of the structure of a C2PSA module that integrates attention and multi-scale according to an embodiment of the present invention;

[0041] Figure 6 This is a schematic diagram of the Path Aggregation Feature Pyramid Network (PAFPN) provided in one embodiment of the present invention;

[0042] Figure 7This is a flowchart of an outdoor pipeline corrosion detection method based on UAV visual recognition provided in another embodiment of the present invention;

[0043] Figure 8 This is a schematic diagram of the structure of an outdoor pipeline corrosion detection device based on UAV visual recognition provided in one embodiment of the present invention;

[0044] Figure 9 This is a schematic diagram of an electronic device provided in one embodiment of the present invention. Detailed Implementation

[0045] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0046] It should be noted that although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first," "second," etc., in the specification, claims, and the following drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0047] In this embodiment of the invention, the terms "furthermore," "exemplarily," or "optionally" are used as examples, illustrations, or descriptions and should not be construed as being more preferred or advantageous than other embodiments or designs. The use of the terms "furthermore," "exemplarily," or "optionally" is intended to present the relevant concepts in a specific manner.

[0048] To facilitate a more convenient description of the working principle of the embodiments of the present invention, the following introduction of relevant technical scenarios is given first.

[0049] Factory outdoor pipelines are widely distributed in complex environments such as elevated, winding, and inclined sections. Their long-term exposure to the outdoor environment makes them susceptible to corrosion from wind and rain, leading to rust and potentially causing safety risks such as leaks and explosions. For example, oil pipelines crossing river valleys or steep slopes are often erected using bridge supports or cables, making them vulnerable to wind, rain, and sun exposure, resulting in corrosion. Engineering pipelines (such as water and natural gas pipelines) crossing urban obstacle zones (such as railways and highways) may use culverts, which are also prone to corrosion over time. Currently, common corrosion detection methods include manual inspection, laser scanning, and infrared thermal imaging. These methods suffer from problems such as long inspection cycles, limited accuracy, inconvenient observation angles, and high costs. Especially for irregular structures such as bends and inclined pipes, traditional detection models cannot accurately reconstruct their true position and angle, easily leading to missed detections or misjudgments. Therefore, improving the accuracy and efficiency of outdoor pipeline corrosion detection has become an urgent technical problem to be solved.

[0050] Based on this, the present invention provides a method and related apparatus for detecting corrosion of outdoor pipelines based on UAV visual recognition. The method includes: obtaining corrosion images of outdoor pipelines through UAV patrol and photography of factory outdoor pipelines; inputting these images into a target detection model with a rotating bounding box for corrosion detection; the target detection model employing an integrated design of backbone network, neck network, and detection head; the backbone network extracting features from the corrosion images to obtain multi-scale features; the neck network fusing the multi-scale features to obtain fused features, improving the multi-scale feature fusion capability; and the detection head using joint optimization of rotating bounding box regression and classification to classify and predict the fused features, outputting outdoor pipeline corrosion detection results including corrosion area location information and detection boxes with confidence scores, significantly improving the recognition accuracy of curved, tilted, or irregular corrosion areas. Based on the outdoor pipeline corrosion detection results, a detection report visually displaying the pipeline corrosion distribution is generated and sent to the user terminal for visualization, facilitating quick understanding of the detection results. The entire detection process is efficient, fast, and highly accurate. Based on this, the embodiments of the present invention can comprehensively improve the accuracy and efficiency of outdoor pipeline corrosion detection.

[0051] The embodiments of the present invention will be further described below with reference to the accompanying drawings.

[0052] like Figure 1A As shown, Figure 1A This is a flowchart of an outdoor pipeline corrosion detection method based on UAV visual recognition provided in an embodiment of the present invention. The outdoor pipeline corrosion detection method based on UAV visual recognition may include, but is not limited to, steps S101 to S103.

[0053] Step S101: Obtain images of outdoor pipe corrosion, wherein the images of outdoor pipe corrosion are obtained by taking pictures of the factory's outdoor pipes by drone.

[0054] Step S102: Input the outdoor pipeline corrosion image into the pre-trained target detection model for corrosion detection to obtain the outdoor pipeline corrosion detection result. The target detection model includes a backbone network, a neck network, and a detection head. The backbone network is used to extract features from the outdoor pipeline corrosion image to obtain multi-scale features. The neck network is used to fuse the multi-scale features to obtain fused features. The detection head is used to classify the fused features and perform bounding box regression prediction to obtain the outdoor pipeline corrosion detection result. The outdoor pipeline corrosion detection result includes corrosion area location information and detection boxes with confidence.

[0055] Step S103: Generate an inspection report based on the outdoor pipeline corrosion inspection results, and send the inspection report to the user terminal for visual display.

[0056] Understandably, the acquisition of images of outdoor pipe corrosion involves using a drone to fly along the pipes outside the factory, with an onboard camera capturing images. The captured images are then immediately sent to a pre-trained target detection model for corrosion detection.

[0057] It is understandable that, such as Figure 1B As shown, images of outdoor pipeline corrosion are input into a target detection model with rotating bounding boxes for corrosion detection. The target detection model adopts an integrated design of backbone network, neck network, and detection head. The backbone network extracts features from the outdoor pipeline corrosion images to obtain multi-scale features. The neck network fuses the multi-scale features to obtain fused features, improving the multi-scale feature fusion capability. The detection head uses joint optimization of rotating bounding box regression and classification to classify and predict the fused features, outputting outdoor pipeline corrosion detection results including corrosion area location information and detection boxes with confidence scores, significantly improving the recognition accuracy of curved, tilted, or irregular corrosion areas.

[0058] Understandably, during the training phase, the object detection model first needs to construct a training dataset. Specifically, this involves two aspects of dataset collection. Firstly, this invention utilizes advanced web crawling technology to collect a large number of pipe corrosion images from the internet, and then performs preliminary processing and filtering. By removing highly similar images, junk data, and blurry or severely occluded images, this invention obtains a relatively clean and effective pipe corrosion image dataset. Secondly, an industrial-grade drone is used to patrol along the pipes outdoors in the factory. The patrol is divided into four flights, each covering 1 / 4 of the pipe area, resulting in full coverage. Regular aerial photography sampling is performed on various pipes within the covered area to collect real-world pipe corrosion image data.

[0059] After the dataset was collected, all images underwent uniform preprocessing. All images were re-cropped to a fixed size (640×640) and enhanced with geometric transformations: random rotation (0 to 360 degrees), scaling (0.8 to 1.2 times), translation (±20%), and mirroring were applied to simulate various postures of pipe corrosion in a real environment. Illumination enhancement was achieved using techniques such as Gamma correction (0.5 to 2.0), random brightness adjustment (±30%), and contrast variation (0.7 to 1.3) to simulate different lighting conditions. Environmental interference enhancement included adding Gaussian noise (σ = 0 to 0.1), random occlusion (10 to 30% area), and motion blur (3 to 15 pixels). These uniform preprocessing operations not only improve training stability but also help reduce computational resource consumption and increase training efficiency.

[0060] Next, the rotated bounding box is labeled using a nine-parameter format (cls, x1, y1, X2, Y2, X3, Y3, X4, Y4). CLS is either 0 or 1, with a value of 1 indicating corrosion and 0 indicating no corrosion. x1 and Y1 represent the first coordinate of the rusted area rotated clockwise, X2 and Y2 represent the second coordinate, X3 and Y3 represent the third coordinate, and X4 and Y4 represent the fourth coordinate. 30% of the images in the entire dataset are selected for initial training. These images, representing areas with concentrated pipe corrosion, are labeled using advanced AI annotation software with high precision and efficiency, followed by manual correction. The model is trained using these 30% of images, and then used for inference to predict the labels for the remaining 70% of the data. Finally, the remaining 70% of the annotation results are rigorously verified manually to correct erroneous labels, generating a high-quality model training annotation file. This significantly reduces the workload of purely manual annotation. Based on this, the collected image data, processed as described above, forms a high-quality rotated bounding box corrosion detection dataset.

[0061] Understandably, during the detection phase, the input image (640×640) is first fed into the backbone network to extract multi-scale features. The backbone network uses a lightweight module combination, including standard convolution, residual module C3K2, channel and spatial attention mechanism CBAM, spatial pyramid pooling SPPF, and C2PSA module that integrates attention and multi-scale features.

[0062] Convolution uses a 3x3 kernel and a stride of 2 to automatically learn and extract effective feature representations from the original image. These features are crucial for object detection. The convolution operation processes local regions of the image through a sliding window, capturing local features such as edges and textures.

[0063] like Figure 2As shown, the residual module C3K2 is constructed from C3K, the bottleneck module, and the CBS module. The CBS module consists of convolution, batch normalization, and an activation function (SiLU). The batch normalization layer (BatchNorm) normalizes the output of the convolutional layer, accelerating the training process and improving the model's convergence speed. It standardizes the features by learning the mean and variance of each feature, and then applies learnable scaling and offset parameters. The batch normalization calculation formula is:

[0064]

[0065] Where μ is the mean of the batch data and σ is the variance. 2 ε is a local minimum value used to prevent the denominator from being zero; γ and β are learnable parameters used to scale and translate the normalized data. The input x is processed by the batch normalization formula to obtain the output y. This batch normalization layer makes the feature distribution of the convolutional layer output more stable, which helps the subsequent activation function to play a better role.

[0066] The SiLU activation function is a non-linear activation function used to increase the non-linear expressiveness of a model. The formula for the SiLU function is: f(x) = x*sigmoid(x). It approximates a linear function when x>0 and gradually approaches 0 when x<0, which helps to alleviate the gradient vanishing problem.

[0067] It is understandable that the structural diagram of CBAM is as follows: Figure 3As shown, CBAM is used to enhance the performance of convolutional neural networks. By introducing channel attention and spatial attention, it improves performance without increasing network complexity, enabling it to more effectively focus on important channel and spatial location information, thereby improving performance when detecting rust targets. The goal of the channel attention module is to enhance the feature representation of each channel. First, global max pooling and global average pooling are performed on each channel of the input feature map, resulting in two channel description vectors, representing the global maximum feature and average feature of each channel, respectively. These two vectors are then input into a shared fully connected layer to learn the attention weights for each channel. Next, a sigmoid activation function is used to ensure that the attention weights are between 0 and 1. Finally, the obtained attention weights are multiplied by each channel of the original feature map to obtain the attention-weighted channel feature map. The spatial attention module is used to enhance the feature representation of each spatial location in the feature map. First, inter-channel max pooling and average pooling are performed on the feature map, resulting in two feature maps that are spatially identical to the original feature map and have one channel. Then, the two feature maps are concatenated along the channel dimension, and a 7×7 convolution operation is used to fuse the two feature maps into a spatial attention feature map. Finally, the attention weights are restricted to between 0 and 1 by the Sigmoid activation function and multiplied with the original feature map to obtain the spatial attention feature map.

[0068] It is understandable that the SPPF structure diagram is as follows: Figure 4 As shown, SPPF (Spatial Pyramid Pooling) extracts and encodes features from images at different scales. It can rescale an input image of any size to a fixed size and generate a fixed-length feature vector. Since the output size of traditional pooling layers is tied to the input size, but fully connected layers require specified inputs for classification, a method is needed to allow the neural network to obtain a fixed-dimensional output at a certain layer. SPPF is thus used to generate fixed-length feature vectors for images of any size.

[0069] Understandably, the structure diagram of C2PSA is as follows: Figure 5As shown, the C2PSA module is an enhanced feature extraction module that combines multi-scale convolutional features and a channel-wise attention mechanism, suitable for detecting targets with varying scales and orientations in complex scenes. It combines a cross-layer fusion structure and the PSA attention mechanism to improve multi-scale feature extraction capabilities. It performs particularly well in object detection in complex scenes, especially when handling multi-scale objects. The PSA mechanism extracts multi-scale features using various convolutional kernels (3x3, 5x5, 7x7, etc.). Convolution operations with different kernels are performed in parallel, and then the feature maps are concatenated and weighted for each feature channel. Finally, the generated attention weights are applied to each feature map, achieving channel-wise weighting and increasing attention to important features.

[0070] Understandably, the neck network bears the crucial task of connecting the backbone network and the detection head. Its core function is to fuse multi-scale feature maps, further enhancing semantic and spatial information, enabling the detection head to simultaneously capture key global and local features, making it particularly suitable for small target detection and rotating target recognition in complex scenes. This allows for more accurate identification of rusted areas. This invention employs an improved PAFPN structure in the neck network design, such as... Figure 6 As shown, by combining lightweight convolution, attention mechanism and cross-layer connection, a balance between feature enhancement and fusion is achieved, which meets the detection needs of complex pipeline corrosion areas captured by UAV.

[0071] Feature Pyramid Network (FPN) upsamples and fuses deep semantic features from top to bottom to enhance the semantic representation of shallow features. However, the original FPN fails to effectively integrate semantic and positional information across layers. The PAFPN structure adds an extra round of multi-scale downsampling to the original FPN and fuses semantic features from the same layer. This way, shallow features pass through the original FPN, are laterally connected to N2, and then aggregated from N2 along a bottom-up path to the top layer N5. With fewer than 10 layers, it better preserves shallow feature information while better integrating multi-scale information.

[0072] like Figure 6 P5 = C5, P4 = Upsample(P5) + C4, P3 = Upsample(P4) + C3, P2 = Upsample(P3) + C2; N2 = P2, N3 = Conv(Down(N2) + P3), N4 = Conv(Down(N3) + P4), N5 = Conv(Down(N4) + P5). The upsampling function Upsample uses bilinear interpolation. The formula is:

[0073]

[0074] Where V(x,y) is the value of a pixel in the target image, I(x1,y1), I(x2,y1), I(x1,y2), and I(x2,y2) are the values ​​of four adjacent pixels in the source image, and (x1,y1) to (x2,y2) is a 2×2 pixel region in the source image containing the position corresponding to the target pixel. Bilinear interpolation finds the four nearest pixels in the source image to the target pixel (x,y) at the corresponding coordinates, and calculates the RGB value of that point using formula (2). By calculating all points in the target image in this way, upsampling is completed.

[0075] This neck structure combines downsampling and upsampling operations, fully integrating multi-scale and spatial location information to enhance the structural detection capability for curved and tilted corroded areas.

[0076] Understandably, the detection head module is responsible for classifying the fused feature map and regressing and predicting the five-parameter rotated bounding box, outputting the location information of the rusted area in (x,y,w,h,θ) format and the confidence score of the predicted bounding box. This invention adopts a single-stage structure, eliminating the need for generating region proposal boxes and performing secondary regression and classification as in a two-stage approach. Instead, it directly performs classification and regression on the feature map, simplifying the process, improving real-time processing speed, and playing a crucial role in rapidly identifying pipeline rust. The model of this invention introduces a five-dimensional rotated bounding box prediction mechanism (x,y,w,h,θ), which can accurately fit rusted areas with various angular distributions. Combined with a lightweight structure and a multi-scale attention enhancement mechanism, it possesses excellent performance for industrial deployment.

[0077] The image is divided into a 20x20 grid. Each grid cell generates three bounding boxes, detecting the center point of objects within that cell. Each bounding box is assigned a confidence score. These scores reflect the model's confidence in the box containing rust and the accuracy of its prediction. A cell without rust has a confidence score of zero, while a cell with rust has a confidence score equal to the intersection-over-union (IoU) ratio between the predicted and ground truth bounding boxes.

[0078] Each bounding box consists of six predictions: x, y, w, h, θ, and a confidence score. The (x, y) coordinates represent the center of the box relative to the boundary of the grid cell. w and h represent the width and height, which are predicted relative to the entire image, and θ is the longer side le135 in OBB format.

[0079] During head training, the loss function for bounding box regression is Smooth L1. The Smooth L1 loss is calculated for each offset (dx, dy, dw, dh, dθ) and its corresponding true value error, and then summed. Smooth L1 handles angular differences well. The Smooth L1 loss function is as follows:

[0080]

[0081] The classification task is a binary classification problem, with categories of rust and non-rust. The category labels are: rust (ti=1) and non-rust (ti=0). Supervised learning is performed using the cross-entropy loss function. Cross-entropy loss function:

[0082]

[0083] Where p is the predicted label and t is the true label.

[0084] The corrosion detection network uses multi-task processing for detection, simultaneously training classification and rotational regression. The following multi-task loss function is used: Equation (5)

[0085]

[0086] Here, λ1 and λ2 are parameters balancing the weights of classification and rotation regression. u is the rotated ground truth bounding box, and u* is the rotated predicted bounding box. The detection head, backbone, and neck networks are jointly trained end-to-end. During backpropagation, the classification loss and rotation regression loss are combined to train the entire network. The network model parameters are continuously optimized through the backpropagation algorithm to reduce the loss value and improve the model's detection performance.

[0087] In one embodiment, such as Figure 7 As shown, a drone flies along the pipeline outside the factory, its onboard camera capturing images. The captured images are immediately sent to a model for corrosion detection. The results are post-processed, employing non-maximum suppression to suppress low-confidence detection boxes and obtain high-confidence boxes. The detection results are then visualized by drawing detection boxes on the original image. The color and style of the detection boxes are differentiated based on the presence of corrosion: solid red boxes indicate corrosion, while dashed blue boxes indicate other debris. Information is also labeled on the original image, including the corrosion label name, confidence value, and geographic coordinates, visually displaying the distribution of pipeline corrosion. A detection report is generated, including the geographic location of the detected area, the shooting time, the quantity and density of identified pipeline corrosion, and detection accuracy, presented in a combination of charts and text descriptions, such as bar charts and line graphs. The warning results are transmitted back to the user via a 5G wireless network for quick access to the detection results. The warning data is subsequently transmitted to a data storage server for permanent storage.

[0088] Compared with the prior art, the present invention has at least the following beneficial effects:

[0089] High-precision rotating target detection: The introduction of a target detection algorithm with a rotating frame can accurately fit rusted areas of various angles and irregular shapes. It is particularly suitable for curved, tilted or obstructed pipe surfaces, significantly improving detection accuracy and robustness, and solving the problems of missed detection and misjudgment in traditional horizontal frame detection.

[0090] End-to-end lightweight deployment: It adopts an integrated design of backbone-neck-detector head. The backbone network integrates C3K2, CBAM attention mechanism and C2PSA module to achieve a balance between feature extraction capability and computational efficiency. The improved PAFPN neck structure enhances multi-scale feature fusion capability. The detector head uses rotation box regression and classification joint optimization. The overall structure is lightweight and can be trained end-to-end, making it suitable for deployment on resource-constrained UAV platforms to achieve efficient real-time inference.

[0091] Multi-task joint optimization mechanism: The detection head simultaneously performs binary classification (corrosion / non-corrosion) and rotating box localization tasks, and introduces a multi-task loss function with balanced weights, which significantly enhances the network's ability to express and distinguish corrosion targets, making it particularly suitable for detecting complex corrosion morphology in real industrial scenarios.

[0092] The remote intelligent inspection and early warning functions are complete: combined with the drone autonomous patrol system, it can realize fixed-point, timed, and directional sampling and inspection of the factory's outdoor pipelines; the inspection results are used to generate inspection maps through non-maximum suppression and visualization rendering, and form a graphic inspection report. The report is fed back to the user terminal in real time through the 5G network, providing detailed early warning information such as location coordinates, confidence level, and corrosion distribution map.

[0093] Wide range of industrial applications: This invention is widely applicable to the outdoor pipeline inspection needs of industries such as petroleum, chemical, power, and natural gas. It performs particularly well in areas that are difficult to inspect manually, such as elevated, inclined, and curved pipelines. It has high reliability, high level of automation, and good prospects for engineering promotion. It is especially suitable for long-distance pipeline corrosion detection.

[0094] Based on this, this invention utilizes an image acquisition and recognition system mounted on a drone, combined with a rotating bounding box target detection algorithm, to achieve real-time detection, localization, and early warning of rusted areas of various angles and shapes. This invention innovatively introduces rotating bounding box modeling and channel and spatial attention mechanisms, significantly improving the recognition accuracy of curved, tilted, or irregular rusted areas. The entire system, from image acquisition, data augmentation, rotating bounding box training, post-processing filtering, early warning determination to result visualization, constructs a complete technology chain, possessing high precision, high robustness, and high deployment efficiency. It can be widely applied to rust inspection scenarios in pipeline equipment in industries such as chemical, power, and petrochemical.

[0095] In addition, such as Figure 8As shown, one embodiment of the present invention also discloses an outdoor pipeline corrosion detection device based on UAV visual recognition, the device comprising:

[0096] The image acquisition module is used to acquire images of outdoor pipeline corrosion, which are obtained by drones taking pictures of the factory's outdoor pipelines during patrols.

[0097] The detection module is used to input outdoor pipeline corrosion images into a pre-trained target detection model for corrosion detection, and obtain outdoor pipeline corrosion detection results. The target detection model includes a backbone network, a neck network, and a detection head. The backbone network is used to extract features from the outdoor pipeline corrosion images to obtain multi-scale features. The neck network is used to fuse the multi-scale features to obtain fused features. The detection head is used to classify the fused features and perform bounding box regression prediction to obtain outdoor pipeline corrosion detection results. The outdoor pipeline corrosion detection results include corrosion area location information and detection boxes with confidence scores.

[0098] The output module is used to generate an inspection report based on the results of outdoor pipeline corrosion detection and send the inspection report to the user terminal for visualization.

[0099] The outdoor pipeline corrosion detection device based on UAV visual recognition in this embodiment of the invention is used to execute the outdoor pipeline corrosion detection method based on UAV visual recognition in the above embodiment. Its specific processing procedure is the same as that of the outdoor pipeline corrosion detection method based on UAV visual recognition in the above embodiment, and will not be described in detail here.

[0100] In addition, such as Figure 9 As shown, one embodiment of the present invention also discloses an electronic device, including: at least one processor 210; at least one memory 220 for storing at least one program; when the at least one program is executed by the at least one processor 210, it implements the outdoor pipeline corrosion detection method based on UAV visual recognition as in any of the preceding embodiments.

[0101] In addition, one embodiment of the present invention discloses a computer-readable storage medium storing computer-executable instructions for performing an outdoor pipeline corrosion detection method based on UAV visual recognition as described in any of the preceding embodiments.

[0102] The system architecture and application scenarios described in the embodiments of this invention are for the purpose of more clearly illustrating the technical solutions of the embodiments of this invention, and do not constitute a limitation on the technical solutions provided by the embodiments of this invention. As those skilled in the art will know, with the evolution of system architecture and the emergence of new application scenarios, the technical solutions provided by the embodiments of this invention are also applicable to similar technical problems.

[0103] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0104] In hardware implementations, the division between functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, as is known to those skilled in the art, communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0105] The terms “component,” “module,” “system,” etc., used in this specification are used to refer to computer-related entities, hardware, firmware, combinations of hardware and software, software, or software in execution. For example, a component can be, but is not limited to, a process running on a processor, a processor, an object, an executable file, an execution thread, a program, or a computer. As illustrated, applications running on computing devices and computing devices can both be components. One or more components may reside in a process or execution thread, and components may be located on a single computer or distributed among two or more computers. Furthermore, these components can be executed from various computer-readable media on which various data structures are stored. Components can communicate, for example, via local or remote processes based on signals having one or more data packets (e.g., data from two components interacting with another component between a local system, a distributed system, or a network, such as the Internet interacting with other systems via signals).

Claims

1. A method for detecting corrosion of outdoor pipelines based on UAV visual recognition, comprising: Images of outdoor pipe corrosion are obtained by taking pictures of the factory's outdoor pipes using a drone. The outdoor pipeline corrosion image is input into a pre-trained target detection model for corrosion detection to obtain the outdoor pipeline corrosion detection result. The target detection model includes a backbone network, a neck network, and a detection head. The backbone network is used to extract features from the outdoor pipeline corrosion image to obtain multi-scale features. The neck network is used to fuse the multi-scale features to obtain fused features. The detection head is used to classify the fused features and perform bounding box regression prediction to obtain the outdoor pipeline corrosion detection result. The outdoor pipeline corrosion detection result includes corrosion area location information and detection boxes with confidence scores. An inspection report is generated based on the outdoor pipeline corrosion detection results, and the inspection report is sent to the user terminal for visual display.

2. The method according to claim 1, characterized in that, The training method for the object detection model includes: Batch acquisition of publicly available, free images of pipe corrosion from the internet using web crawlers; The images of internet pipeline corrosion are filtered to obtain internet pipeline corrosion image data; Drones were used to take aerial photos of the factory's outdoor pipelines at regular intervals to obtain real-world images of pipeline corrosion. Based on the internet pipeline corrosion image data and the real-world pipeline corrosion image data, an image dataset is obtained; The image dataset is preprocessed to obtain the target image dataset; The target image dataset is annotated with a rotating bounding box using a preset parameter format to obtain a rotating bounding box corrosion detection dataset. The target detection model is trained based on the rotating frame corrosion detection dataset to obtain the trained target detection model.

3. The method according to claim 2, characterized in that, The preprocessing of the image dataset to obtain the target image dataset includes: All images in the image dataset are cropped according to a preset size to obtain the first image dataset; The first image dataset is subjected to geometric transformation enhancement to obtain the second image dataset, wherein the geometric transformation enhancement includes random rotation, scaling, translation and mirror flipping; The second image dataset is subjected to illumination enhancement to obtain a third image dataset, wherein the illumination enhancement includes gamma correction, random brightness adjustment and contrast variation; The third image dataset is subjected to environmental interference enhancement to obtain the target image dataset, wherein the environmental interference enhancement includes adding Gaussian noise, random occlusion, and motion blur.

4. The method according to claim 1, characterized in that, The backbone network includes standard convolution, residual module C3K2, channel and spatial attention mechanism CBAM, spatial pyramid pooling SPPF, and a C2PSA module that integrates attention and multi-scale features. The standard convolution uses a 3*3 kernel and a stride of 2. The residual module C3K2 is constructed from C3K, a bottleneck module, and a CBS module. The CBS module consists of convolutional layers, batch normalization layers, and SiLU activation functions. The spatial pyramid pooling SPPF is used to extract and encode features from the image at different scales and generate fixed-length feature vectors. The C2PSA module that integrates attention and multi-scale features is used to extract multi-scale features using multiple convolutional kernels. The convolution operations of different kernels are performed in parallel. The feature maps are then concatenated, and the feature channels are weighted. Finally, the generated attention weights are applied to each feature map to achieve point-by-point weighting of the channels.

5. The method according to claim 4, characterized in that, The channel and spatial attention mechanism CBAM includes a channel attention module and a spatial attention module. The channel attention module performs global max pooling and global average pooling operations on each channel of the input feature map to obtain two channel description vectors, representing the global maximum feature and average feature of each channel, respectively. These two channel description vectors are then input into a shared fully connected layer to learn the attention weights for each channel. A sigmoid activation function is then used to ensure that the attention weights are between 0 and 1. Finally, the obtained attention weights are multiplied by each channel of the original feature map to obtain the attention-weighted channel feature map. The spatial attention module performs inter-channel max pooling and average pooling on the channel feature map to obtain two feature maps with the same spatial dimension as the original feature map and a channel count of 1. These two feature maps are then concatenated along the channel dimension and fused into a spatial attention feature map using a 7×7 convolution operation. Finally, a sigmoid activation function is used to restrict the attention weights to between 0 and 1, and the attention weights are multiplied by the original feature map to obtain the spatially attention-weighted spatial feature map.

6. The method according to claim 1, characterized in that, The neck network adopts the Path Aggregation Feature Pyramid Network (PAFPN) structure.

7. The method according to claim 1, characterized in that, The training method for the detection head includes: Determine the classification loss function and the corresponding classification weight parameters; Determine the loss function and corresponding rotation regression weight parameters for the rotation box regression; The detection head is trained using the classification loss function, the classification weight parameters, the loss function of the rotated bounding box regression, and the rotation regression weight parameters to obtain the trained detection head.

8. An outdoor pipeline corrosion detection device based on UAV visual recognition, characterized in that, The device includes: The image acquisition module is used to acquire images of outdoor pipeline corrosion, wherein the images of outdoor pipeline corrosion are obtained by drones taking pictures of the factory's outdoor pipelines during patrols. The detection module is used to input the outdoor pipeline corrosion image into a pre-trained target detection model for corrosion detection, and obtain the outdoor pipeline corrosion detection result. The target detection model includes a backbone network, a neck network, and a detection head. The backbone network is used to extract features from the outdoor pipeline corrosion image to obtain multi-scale features. The neck network is used to fuse the multi-scale features to obtain fused features. The detection head is used to classify the fused features and perform bounding box regression prediction to obtain the outdoor pipeline corrosion detection result. The outdoor pipeline corrosion detection result includes corrosion area location information and detection boxes with confidence scores. The output module is used to generate an inspection report based on the outdoor pipeline corrosion detection results and send the inspection report to the user terminal for visualization.

9. An electronic device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, when the processor executes the computer program, it implements the outdoor pipeline corrosion detection method based on UAV visual recognition as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing computer-executable instructions for performing the outdoor pipeline corrosion detection method based on UAV visual recognition as described in any one of claims 1 to 7.

Images