Pipeline defect detection method and device based on improved YOLOv8 algorithm

Abstract

The invention discloses a pipeline defect detection method and device based on an improved YOLOv8 algorithm, and the method comprises the steps: analyzing a pipe segment video, and obtaining an initial pipe segment image; marking the initial pipe section picture to construct a pipe network defect data set; performing enhancement processing on the initial pipe segment picture to obtain a target pipe segment picture; updating the initial network structure of the initial pipeline defect detection model, and constructing a target loss function to obtain a target pipeline defect detection model; according to the pipe network defect data set, training the target pipeline defect detection model; and obtaining a detection result according to the trained target pipeline defect detection model and the target pipe section picture. The method can improve the accuracy of pipeline defect detection, and can be widely applied to the technical field of pipeline detection.

Description

Technical Field

[0001] The present invention relates to the technical field of pipeline detection, and in particular to a pipeline defect detection method and device based on an improved YOLOv8 algorithm. Background Art

[0002] In the field of pipeline maintenance and inspection, especially for underground drainage pipelines or industrial pipelines, as the service time increases, the inner wall of the pipeline is prone to defects such as misalignment, cracks, corrosion, etc. Traditional pipeline defect detection methods usually rely on manual detection or semi-automatic detection, which has low work efficiency and it is difficult to ensure the accuracy and consistency of the detection results.

[0003] In recent years, with the development of computer vision and deep learning technology, deep learning-based target detection algorithms have gradually been applied to pipeline defect detection. Among them, the YOLO (You Only Look Once) series of algorithms have become one of the preferred models in the field of pipeline defect detection due to their efficient detection speed and high detection accuracy. YOLOv8 is the latest version of the YOLO series, with better performance and higher detection accuracy, and can simultaneously complete the classification and positioning tasks of targets in a single inference. However, despite the excellent performance of YOLOv8 in general target detection tasks, there are still some problems and limitations when it is directly applied to pipeline defect detection tasks. The network structure of YOLOv8 is relatively complex, especially when processing high-resolution images and videos, the amount of calculation is large, and the required hardware resources are more, making it difficult to run efficiently in resource-constrained environments such as mobile devices or embedded systems. In addition, the existing YOLOv8 model is not robust enough in processing degraded images. When processing small targets or overlapping targets, the detection performance of YOLOv8 decreases, and it is easy to miss or misdetect. Summary of the invention

[0004] In view of this, the main purpose of the embodiments of the present invention is to provide a pipeline defect detection method and device based on an improved YOLOv8 algorithm, in order to solve at least one of the problems of the prior art. The present invention can improve the accuracy of pipeline defect detection.

[0005] To achieve the above object, an embodiment of the present invention provides a pipeline defect detection method based on an improved YOLOv8 algorithm, the method comprising the following steps: Analyze the pipe segment video to obtain the initial pipe segment image; By annotating the initial pipe section image, a pipe network defect data set is constructed; Performing enhancement processing on the initial pipe section image to obtain a target pipe section image; The initial network structure of the initial pipeline defect detection model is updated, and a target loss function is constructed to obtain a target pipeline defect detection model; Training the target pipeline defect detection model according to the pipeline network defect data set; A detection result is obtained based on the trained target pipeline defect detection model and the target pipe section image.

[0006] In some embodiments, before the step of parsing the pipe segment video to obtain the initial pipe segment image, the following steps are also included: Reading the pipe network assessment report, extracting information from the pipe network assessment report, and obtaining the pipe section name; According to the pipe segment name, the corresponding pipe segment video is searched and collected.

[0007] In some embodiments, the constructing of the pipeline network defect dataset by annotating the initial pipeline segment image comprises the following steps: Identifying defect locations and defect types in the initial pipe segment image; Annotate the initial pipe segment image according to the defect position and the defect type to obtain an annotation result; According to the labeling results, the pipeline network defect dataset is constructed.

[0008] In some embodiments, the enhancing process of the initial pipe segment image to obtain the target pipe segment image comprises the following steps: Inputting the initial pipe segment image into a CPA-Enhancer enhancement module to generate a target prompt for the initial pipe segment image; Obtaining input features of the initial pipe segment image; The input feature is interacted with the target prompt to obtain the target pipe segment image.

[0009] In some embodiments, the step of inputting the initial pipe segment image into a CPA-Enhancer enhancement module to generate a target prompt for the initial pipe segment image includes the following steps: Input the initial pipe segment image into the CPA-Enhancer enhancement module, and generate an initial prompt after passing through the first decoding layer; A deconvolution operation is performed on the initial prompt to obtain the target prompt.

[0010] In some embodiments, the step of interacting the input feature with the target prompt to obtain the target pipe segment image includes the following steps: Obtaining channel weights and spatial weights of the input features; The input feature, the channel weight and the spatial weight are fused to obtain an enhanced feature; The enhanced feature is interactively processed with the target prompt to obtain the target pipe segment image.

[0011] In some embodiments, the updating of the initial network structure of the initial pipeline defect detection model comprises the following steps: Get the universal inverted bottleneck module, neural architecture search technology, and mobile multi-query attention mechanism; According to the universal inverted bottleneck module, the neural architecture search technology and the mobile version of the multi-query attention mechanism, the initial network structure is updated to obtain a target network structure.

[0012] In some embodiments, constructing the target loss function comprises the following steps: Preset quality labels and non-negative parameters; Obtaining an absolute distance between the predicted value and the quality label; Obtaining an adjustment factor according to the absolute distance and the non-negative parameter; The target loss function is obtained according to the adjustment factor, the quality label and the predicted value.

[0013] To achieve the above object, another aspect of an embodiment of the present invention provides a pipeline defect detection device based on an improved YOLOv8 algorithm, the device comprising: The first module is used to parse the pipe segment video to obtain the initial pipe segment image; The second module is used to construct a pipeline network defect data set by annotating the initial pipeline segment image; The third module is used to enhance the initial pipe section image to obtain a target pipe section image; The fourth module is used to update the initial network structure of the initial pipeline defect detection model and construct a target loss function to obtain a target pipeline defect detection model; A fifth module is used to train the target pipeline defect detection model according to the pipeline network defect data set; The sixth module is used to obtain the detection result according to the trained target pipeline defect detection model and the target pipe section image.

[0014] To achieve the above-mentioned purpose, another aspect of an embodiment of the present invention provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the aforementioned pipeline defect detection method based on the improved YOLOv8 algorithm is implemented.

[0015] To achieve the above-mentioned purpose, another aspect of an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the pipeline defect detection method based on the improved YOLOv8 algorithm described above is implemented.

[0016] To achieve the above object, another aspect of an embodiment of the present invention provides a computer program product or a computer program, the computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the aforementioned pipeline defect detection method based on the improved YOLOv8 algorithm.

[0017] The embodiments of the present invention include at least the following beneficial effects: the present invention provides a pipeline defect detection method and device based on the improved YOLOv8 algorithm, the scheme obtains an initial pipeline segment image by parsing the pipeline segment video; constructs a pipeline network defect data set by annotating the initial pipeline segment image; performs enhancement processing on the initial pipeline segment image to improve the robustness of the model to images under different environments, and obtains a target pipeline segment image; updates the initial network structure of the initial pipeline defect detection model to reduce the model calculation complexity, and constructs a target loss function to obtain a target pipeline defect detection model; trains the target pipeline defect detection model according to the pipeline network defect data set; obtains a detection result according to the trained target pipeline defect detection model and the target pipeline segment image. The present invention can improve the accuracy of pipeline defect detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0019] Figure 1 It is a flow chart of a pipeline defect detection method based on an improved YOLOv8 algorithm provided by an embodiment of the present invention; Figure 2 is a schematic diagram of the internal structure of the algorithm of the CPA-Enhancer enhancement module provided in an embodiment of the present invention; Figure 3 is a schematic diagram of a universal inverted bottleneck module provided by an embodiment of the present invention; Figure 4Schematic diagram of the overall structure of the improved YOLOv8 algorithm model provided by an embodiment of the present invention; Figure 5 It is a schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of the present invention, and they are only examples of devices and methods consistent with some aspects of the embodiments of the present invention as detailed in the attached claims.

[0021] It should be noted that, although the functional modules are divided in the system schematic diagram and the logical order is shown in the flow chart, in some cases, the steps shown or described may be performed in a different order than the module division in the system or the flow chart. The terms "first / S100" and "second / S200" in the specification and claims and the above-mentioned drawings may be used to describe various concepts in this article, but unless otherwise specified, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of the embodiment of the present invention, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the words "if" and "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determination".

[0022] The terms "at least one", "multiple", "each", "any", etc. used in the present invention, at least one includes one, two or more, multiple includes two or more, each refers to each of the corresponding multiple, and any refers to any one of the multiple.

[0023] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which the present invention belongs. The terms used herein are only for the purpose of describing the embodiments of the present invention and are not intended to limit the present invention.

[0024] In the field of pipeline maintenance and inspection, especially for underground drainage pipelines or industrial pipelines, as the use time increases, the inner wall of the pipeline is prone to defects such as misalignment, rupture, and corrosion. The YOLO (You Only Look Once) series of algorithms has become one of the preferred models in the field of pipeline defect detection due to its efficient detection speed and high detection accuracy. Among them, YOLOv8 is the latest version of the YOLO series, with better performance and higher detection accuracy, and can simultaneously complete the classification and positioning tasks of the target in a single inference. However, the network structure of YOLOv8 is relatively complex, especially when processing high-resolution images and videos, the amount of calculation is large, and the required hardware resources are large, making it difficult to run efficiently in resource-constrained environments such as mobile devices or embedded systems. This is a major bottleneck for real-time pipeline inspection tasks. In actual pipeline inspection tasks, inspection equipment (such as pipeline robots) is easily disturbed by environmental factors during movement, such as lighting changes, equipment jitter, and image blur, resulting in poor quality of the captured videos and images. When processing these degraded images, the existing YOLOv8 model is easily significantly affected in detection accuracy and lacks sufficient robustness. In addition, pipeline defects are usually small in size, and multiple defects may exist at the same time or overlap with each other. When dealing with such small or overlapping targets, YOLOv8's detection performance decreases, and it is easy to miss or misdetect.

[0025] In view of this, if Figure 1 As shown, the embodiment of the present invention also provides a pipeline defect detection method based on the improved YOLOv8 algorithm, which may include but is not limited to steps S100 to S600: Step S100, parsing the pipe segment video to obtain an initial pipe segment image; Step S200, constructing a pipeline network defect data set by annotating the initial pipeline segment image; Step S300, performing enhancement processing on the initial pipe segment image to obtain a target pipe segment image; Step S400, updating the initial network structure of the initial pipeline defect detection model, and constructing a target loss function to obtain a target pipeline defect detection model; Step S500, training the target pipeline defect detection model according to the pipeline network defect data set; Step S600, obtaining a detection result according to the trained target pipeline defect detection model and the target pipe section image.

[0026] Before step S100 in some embodiments, the method further includes reading the pipe network assessment report, extracting information from the pipe network assessment report, and obtaining the pipe segment name; searching and collecting the corresponding pipe segment video according to the pipe segment name. Exemplarily, the pipe network assessment report is batch processed by python code, and information is extracted from the pipe network assessment report containing the assessment information of multiple pipe segments. The extracted text includes the names of the pipe segments, such as "pipe segment A-1", "pipe segment B-2", etc. By extracting the names of the pipe segments, a list of pipe segment names can be obtained, and further, video files matching these pipe segment names can be searched in the file system. This example is only an optional embodiment. In some embodiments, the search method can be adjusted according to the specific file format, directory structure, and naming convention. Error handling, logging, etc. may also need to be considered in information extraction and searching for corresponding videos.

[0027] In step S100 of some embodiments, after the pipeline segment video is collected, the video data is parsed into static images frame by frame. Exemplarily, a video parsing module uses standard video decoding technology to save a picture every 60 frames to ensure that the complete scene in the pipeline is captured. The extracted pictures will be stored in chronological order and provide data support for subsequent processing.

[0028] In some embodiments, step S200 may include but is not limited to steps S210 to S230: Step S210, identifying the defect location and defect type of the initial pipe segment image; Step S220, annotating the initial pipe section image according to the defect position and the defect type to obtain an annotation result; Step S230: construct the pipeline network defect dataset according to the labeling results.

[0029] In some optional embodiments, the defect positions and categories of the extracted initial pipe segment images can be manually annotated directly through Labelimg software, and the annotated defect types include misalignment, rupture, corrosion, etc.

[0030] In some optional embodiments, a defect detection model can also be trained by using deep learning techniques, such as convolutional neural networks (CNN). This model can identify the location and type of defects in an image. During the training process, labeled defect images are required so that the model can learn how to identify different defects. The trained model is applied to the initial pipe segment images, and the model will identify and output the defect location and type of each image. According to the defect location and type identified by the model, the images are annotated using automated tools. For example, LabelImg or other image annotation tools can be used to automatically draw bounding boxes or segment defect areas based on the output of the model. Since there may be errors in automated annotation, the annotation results can be manually reviewed and inaccurate annotations can be corrected later. The annotation results are organized into a standard format. For example, the annotation information of each image can be stored as a JSON file, which contains the location, type and other relevant information of the defect. The organized annotation information is combined with the corresponding image to form a pipeline network defect dataset. This dataset can be used for further model training and verification to improve the accuracy of defect detection.

[0031] In some embodiments, step S300 may include but is not limited to steps S310 to S330: Step S310, inputting the initial pipe segment image into a CPA-Enhancer enhancement module to generate a target prompt for the initial pipe segment image; Step S320, obtaining input features of the initial pipe segment image; Step S330: interacting the input feature with the target prompt to obtain the target pipe segment image.

[0032] In steps S310 to S330 of some embodiments, the CPA-Enhancer adaptive image enhancement algorithm is used to enhance the input initial pipe segment image without knowing the degradation type in advance. The CoT prompt generation submodule (CGM) of the CPA-Enhancer enhancement module generates the target prompt of the initial pipe segment image, and the content-driven prompt block (CPB) of the CPA-Enhancer enhancement module interacts with the input features of the initial pipe segment image and enables the model to adaptively adjust the enhancement strategy according to different degradation types.

[0033] CPA-Enhancer is a plug-and-play enhancement module that can enhance the input image without knowing the degradation type in advance. The core idea is to gradually guide the model to adapt to different degradation types through Chain of Thought (CoT) prompts, thereby dynamically adjusting the enhancement strategy. The design of CPA-Enhancer includes two key modules: CoT Prompt Generation Module (CGM) and Content-Driven Prompt Block (CPB). CGM is responsible for generating prompts containing degradation information, while the content-driven prompt block realizes the interaction between input features and prompts, enabling the model to adaptively adjust the enhancement strategy according to different degradation types.

[0034] In some embodiments, the CPA-Enhancer adaptive image enhancement algorithm is used to enhance the extracted pipe segment images, especially under unknown image degradation conditions such as the bumpy walking and shaking of the detection robot during the shooting process, to improve the robustness of the model for images in different environments, thereby improving the accuracy of defect detection. Image degradation refers to the degradation of image quality during the formation, transmission and recording of the image due to the imperfections of the imaging system, transmission medium and equipment.

[0035] In some embodiments, step S310 may include but is not limited to steps S311 to S312: Step S311, inputting the initial pipe segment image into the CPA-Enhancer enhancement module, and generating an initial prompt after passing through the first decoding layer; Step S312, performing a deconvolution operation on the initial prompt to obtain the target prompt.

[0036] In steps S311 to S312 of some embodiments, during the image enhancement process of CPA-Enhancer, a multi-scale chained thinking prompt is first generated by the CGM module. Figure 2 As shown, the prompt generation process is as follows: In the first decoding layer, the initial prompt is generated ,in, , and Represent the height, width and number of channels of the initial prompt respectively. Multi-scale prompts are generated through a series of layers with deconvolution operations. The specific generation process is:

[0037] in, Represents a 3×3 deconvolution operation; represents the activation function used to control the flow of information; Indicates the current target cue; represents the next target cue generated after the current cue is deconvolved. In this way, the generated CoT cue can gradually encode the degradation characteristics of the image, thus providing guidance for the subsequent enhancement process.

[0038] In step S320 of some embodiments, the input initial pipe segment image is subjected to a series of convolution operations to extract input features of the initial pipe segment image. For example, Figure 2 As shown in the figure, the input initial pipe segment image is processed through a series of 3×3 receptive field attention convolution layers (RAFConv) to extract various input features of the initial pipe segment image, which will be used in the subsequent enhancement process.

[0039] In some embodiments, step S330 may include but is not limited to steps S331 to S333: Step S331, obtaining the channel weight and spatial weight of the input feature; Step S332, fusing the input feature, the channel weight and the spatial weight to obtain an enhanced feature; Step S333: interactively process the enhanced feature and the target prompt to obtain the target pipe segment image.

[0040] In some embodiments, in steps S331 to S333, the input features and target prompts are interacted in the content-driven prompt block of the CPA-Enhancer enhancement module to generate an enhanced feature map, i.e., a target pipe segment image. For example, the channel weights of the input features are first obtained. and spatial weight , respectively representing the importance of each channel and spatial dimension. Among them, the channel weight of the input feature is generated by performing global pooling and 1×1 convolution operations on the input feature; the spatial weight of the input feature can be generated by combining 7×7 convolution with global pooling and maximum pooling. The formula is as follows:

[0041]

[0042] In the formula, represents the channel weight; represents the spatial weight; Represents input features; Represents a 1×1 convolution operation; Represents a 7×7 convolution operation; represents the activation function; Represents a global pooling operation; Represents the max pooling operation.

[0043] Then, the channel weights and spatial weights are fused with the input features to obtain enhanced features, i.e., enhanced features. , the calculation formula is as follows:

[0044] Finally, the enhanced features obtained by fusion With target prompt After interactive processing, the enhanced feature map, i.e., the target pipe segment image, can be obtained. This process is carried out step by step in each decoding layer, such as Figure 2 As shown, the multi-scale adaptability of the features is guaranteed, and the adaptive enhancement of different types of degraded images is achieved through the step-by-step guidance of chain thinking prompts.

[0045] In some embodiments, the step of updating the initial network structure of the initial pipeline defect detection model may include but is not limited to steps S410 to S420: Step S410, obtaining a general inverted bottleneck module, a neural architecture search technology, and a mobile version of a multi-query attention mechanism; Step S420: updating the initial network structure according to the universal inverted bottleneck module, the neural architecture search technology and the mobile version multi-query attention mechanism to obtain a target network structure.

[0046] In steps S410 to S420 of some embodiments, the YOLOv8 model is selected as the initial pipeline defect detection model. In order to reduce the computational complexity and parameter amount of the model, the original YOLOv8 feature network structure is updated and replaced with the MobileNetV4 network. The architecture design of MobileNetV4 continues to optimize the efficiency and performance of the network based on MobileNetV3. MobileNetV4 uses deep separable convolution to reduce the computational complexity of the model. Unlike standard convolution, deep separable convolution divides the convolution operation into two steps: deep convolution and point-by-point convolution. Deep convolution is performed by convolution operation on each input channel separately, and point-by-point convolution recombines the features of each channel through 1×1 convolution. This operation greatly reduces the amount of calculation. On this basis, MobileNetV4 further optimizes the convolution operation to make it more efficient and reduce the computational bottleneck. At the same time, MobileNetV4 optimizes the lightweight modules in the network, such as the structure of the inverted residual block. The inverted residual block greatly reduces the amount of computation by expanding the feature dimension, depthwise separable convolution, and recompressing the feature dimension. Two optional depthwise convolutions are introduced in the inverted bottleneck block, one before the expansion layer and the other between the expansion and projection layers. Although this modification is quite simple, the new module well unifies some important existing blocks, including the original IB block, the Conv Next module, and the feedforward neural block in ViT (Vision Transformer). In addition, the general inverted residual block of an embodiment of the present invention introduces a new variant: the extra depth block. The extra depth block can achieve a cheap increase in network depth and receptive field, providing the combined advantages of the Conv Next module and the IB block. Exemplarily, to obtain the general inverted bottleneck module, refer to Figure 3 , Figure 3 An optional structural diagram of a universal inverted bottleneck module is provided, in which the universal inverted bottleneck module includes a universal inverted residual block 701 with two optional deep convolutions; a universal inverted residual block 702 with additional deep convolutions, an inverted bottleneck block, a Conv Next module, and a feedforward neural network; and an alternative fused inverted residual block 703.

[0047] In some embodiments, neural architecture search (NAS) technology is used to automatically design network architectures. Neural architecture search can automatically generate the optimal network structure based on different hardware requirements (such as mobile phones, embedded devices, etc.), ensuring that the inference speed and accuracy of the model on different platforms are optimally balanced.

[0048] In some embodiments, a mobile multi-query attention mechanism (Mobile MQA) optimized for mobile accelerators is introduced. Compared with the traditional multi-head self-attention mechanism (MHSA), the mobile multi-query attention mechanism provides about 39% inference acceleration on mobile devices. Mobile MQA is an improvement on multi-query attention (MQA), which simplifies the calculation process by sharing the keys and values ​​of multiple heads, thereby greatly improving efficiency. In traditional multi-head self-attention, each head has independent queries, keys, and values. Although this design can effectively capture different features in large models, its high computational overhead is a challenge that cannot be ignored for mobile devices. The embodiment of the present invention uses Mobile MQA to optimize this problem and retains multiple query heads. At the same time, Mobile MQA proposes a mechanism for sharing keys and values ​​between all heads. This sharing not only reduces the amount of calculation, but also significantly reduces the need for memory access, especially when the feature dimension is high and the batch size is small. The visual model on the mobile terminal is usually a hybrid model. During the processing, the attention mechanism is only used in the later stage of low resolution. The feature dimension of these stages is usually high, and the batch size is usually 1. In this case, the existence of multiple query heads is still necessary, but sharing keys and values ​​can significantly reduce the consumption of memory bandwidth, thereby improving computational efficiency. Mobile MQA of an embodiment of the present invention further improves efficiency by introducing spatial reduced attention (SRA). In the traditional attention mechanism, the calculations between queries, keys, and values ​​are symmetrical, while Mobile MQA uses an asymmetric calculation strategy to reduce the computational overhead by reducing the resolution of keys and values ​​while retaining the high resolution of queries. This asymmetric calculation strategy is based on an important observation: in the early stages of the hybrid vision model, there is a strong correlation between spatially adjacent tokens, which is usually captured by convolutional filters. Therefore, in Mobile MQA, by reducing the spatial resolution of keys and values, the model can significantly improve computational efficiency while maintaining high-resolution queries. To achieve this spatial reduction, Mobile MQA replaces the traditional average pooling (AvgPooling) with a 3×3 depth convolution with a stride of 2. This operation not only effectively performs spatial downsampling, but also increases the capacity of the model by introducing deep convolutions, thereby improving feature expression capabilities. This optimization strategy not only maintains the same number of tokens between input and output, but also significantly reduces the amount of calculation, allowing Mobile MQA to run efficiently on limited hardware resources. The calculation formulas involved in the Mobile MQA mechanism include:

[0049]

[0050] In the formula, Represents the output of the mobile version of the multi-query attention mechanism; represents the input feature matrix; Represents the concatenation operation, which concatenates the outputs of multiple attention heads; Represents the 1st to the The output of an attention head; Representative The output of an attention head; Represents the output projection matrix, that is, the spliced ​​multi-head attention output will undergo a linear transformation, the output projection matrix is the parameter of the linear transformation. Through this linear projection, the model can further integrate the outputs of multiple heads; represents the activation function used to transform the input into a probability distribution, ensuring that the sum of the calculated attention weights is 1; Representative The query matrix of the attention heads; represents the bond matrix; Representative value matrix; represents the key vector dimension; stands for spatial reduction, which is achieved by designing a depthwise convolution with a stride of 2, or without using spatial reduction represents the identity function, that is It is an adjustable operation that reduces the spatial size or keeps the input unchanged as needed; Represents a transpose operation.

[0051] By introducing MobileNetV4 to replace the original YOLOv8 feature extraction network, the number of model parameters and computational complexity are significantly reduced. While maintaining detection accuracy, the inference speed and efficiency of the model on mobile or embedded devices are improved. This improvement is not only suitable for small devices, but also improves the scalability of the model, making it perform better in resource-constrained environments.

[0052] In some embodiments, constructing the target loss function may include but is not limited to steps S430 to S460: Step S430, presetting quality labels and non-negative parameters; Step S440, obtaining an absolute distance between the predicted value and the quality label; Step S450, obtaining an adjustment factor according to the absolute distance and the non-negative parameter; Step S460: Obtain the target loss function according to the adjustment factor, the quality label and the predicted value.

[0053] In some embodiments, in steps S430 to S460, by constructing a Quality Focal Loss (QFL) target loss function, the prediction quality of each sample can be modeled, and the weight of the loss function can be adjusted according to the prediction quality. , and defines a continuous value between 0 and 1 to indicate the quality of the prediction. In addition, a non-negative parameter is set Then, get the predicted value and quality labels The absolute distance between them, further, obtain the absolute distance The power is used as a regulating factor to adjust the loss function. The calculation formula of the target loss function is:

[0054] In the formula, Represents the target loss function; Represents a quality label; represents the predicted value; Represents the absolute distance between the predicted value and the quality label, i.e., the adjustment factor; Represents a non-negative parameter, which is used to control the influence of the adjustment factor on the target loss function.

[0055] When the deviation between the predicted value and the quality label is large, the value of the adjustment factor will be relatively large, which means that the model will pay more attention to these difficult-to-predict samples. As the prediction quality improves, that is, the predicted value is close to the quality label, the adjustment factor approaches 0, and the loss of the sample with better prediction will be reduced. The rate of reducing the loss is controlled by the non-negative parameter.

[0056] In some embodiments, the positioning quality of the target (such as the overlap metric between the bounding box and the real object, such as the IoU score) is directly integrated into the classification loss function, and the prediction quality of the sample is modeled through the QFL loss function, and the loss weight is dynamically adjusted. Compared with the traditional Focal Loss, the QFL target loss function not only pays attention to the category of the sample, improves the model's attention to difficult-to-classify samples, but also pays attention to the model's prediction confidence for each sample, so that QFL can more effectively deal with the problems of category imbalance and sample difficulty, especially in complex environments, and can improve the overall detection performance of the model.

[0057] The basic principles of QFL can be divided into three key points: joint representation, continuous label support, and dynamic difficulty adjustment: 1. Joint representation is the core concept of Quality Focal Loss, which can integrate classification scores and positioning quality (such as IoU scores) into a single prediction vector. This representation method effectively solves the separation problem between quality estimation and classification scores in the training and reasoning stages of traditional target detection. Specifically, it allows the model to estimate the positioning quality of each detection box while predicting the classification, thereby providing more accurate ranking scores in non-maximum suppression (NMS) processing and improving detection performance.

[0058] 2. Continuous label support means that in QFL, the output label of the classification is no longer limited to 0 or 1, but can take any continuous value between 0 and 1. These continuous values ​​represent the quality of target positioning, which is usually associated with the intersection over union (IoU) of the true bounding box. In this way, QFL can directly integrate the positioning quality in the loss function, thereby giving greater weight to samples with inaccurate positioning, thereby encouraging the model to learn to predict bounding boxes more accurately.

[0059] 3. Dynamic adjustment of difficulty is one of the important features of QFL, which allows the model to pay more attention to samples that are difficult to classify or locate during training. This is achieved by adjusting a parameter in the loss function, which increases the loss value of the model for samples with higher prediction uncertainty, thereby prompting the model to focus more on these difficult-to-predict samples. This method aims to increase the model's sensitivity to difficult samples and help the model perform more accurate classification and positioning in complex or ambiguous detection scenarios.

[0060] Through the above steps, we can get Figure 4 The improved YOLOv8 model shown, i.e., the target pipeline defect detection model, is trained on the constructed pipeline defect dataset in step S500 of some embodiments. During the training process, a series of hyperparameters can be set to ensure the convergence of the model and the training effect. During the model training process, the SGD optimizer is used to reduce the probability of the model falling into the local optimum. The iteration rounds of model training are set to 300, the input image size is unified to 640×640, the batch size is 4, that is, the number of samples passed to the model in a single batch is 4, the initial learning rate is set to 0.001, and the SGD stochastic gradient optimizer is used, and the momentum factor is 0.937. The entire training process is performed on the GPU to speed up the calculation.

[0061] In some embodiments, after the model training is completed, in order to more comprehensively evaluate the effectiveness of the model in defect detection on the pipeline network disease dataset, the rationality of the target pipeline defect detection model can also be verified by setting up an ablation experiment to further quantify the impact of the improved module on the overall performance of the model. In order to explore the performance of the improved YOLOv8 algorithm in pipeline disease dataset detection, the precision, recall and parameter quantity are used as evaluation indicators to test the improved algorithm.

[0062] In step S600 of some embodiments, the trained target pipeline defect detection model is applied to the real-time scene of the underground pipeline network, and the target pipe segment image is input into the trained target pipeline defect detection model to obtain the detection result.

[0063] The embodiment of the present invention further provides a pipeline defect detection device based on an improved YOLOv8 algorithm, which can implement the above-mentioned pipeline defect detection method based on the improved YOLOv8 algorithm. The device includes: The first module is used to parse the pipe segment video to obtain the initial pipe segment image; The second module is used to construct a pipeline network defect data set by annotating the initial pipeline segment image; The third module is used to enhance the initial pipe section image to obtain a target pipe section image; The fourth module is used to update the initial network structure of the initial pipeline defect detection model and construct a target loss function to obtain a target pipeline defect detection model; A fifth module is used to train the target pipeline defect detection model according to the pipeline network defect data set; The sixth module is used to obtain the detection result according to the trained target pipeline defect detection model and the target pipe section image.

[0064] It can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0065] The embodiment of the present invention further provides an electronic device, which includes a processor and a memory, wherein the memory stores a computer program, and when the processor executes the computer program, the above-mentioned pipeline defect detection method based on the improved YOLOv8 algorithm is implemented. The electronic device can be any intelligent terminal including a tablet computer, a vehicle-mounted computer, etc.

[0066] It can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0067] refer to Figure 5 , Figure 5 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes: The processor 801 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present invention; The memory 802 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 802 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 802, and the processor 801 calls and executes a pipeline defect detection method based on an improved YOLOv8 algorithm according to an embodiment of the present invention; Input / output interface 803, used to implement information input and output; The communication interface 804 is used to realize the communication interaction between the device and other devices. The communication can be realized through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.); A bus 805 that transmits information between various components of the device (e.g., the processor 801, the memory 802, the input / output interface 803, and the communication interface 804); The processor 801 , the memory 802 , the input / output interface 803 and the communication interface 804 are connected to each other in communication within the device via a bus 805 .

[0068] An embodiment of the present invention also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the above-mentioned pipeline defect detection method based on the improved YOLOv8 algorithm.

[0069] It can be understood that the contents of the above method embodiments are all applicable to the present storage medium embodiments, the functions specifically implemented by the present storage medium embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0070] The embodiment of the present invention also provides a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the aforementioned pipeline defect detection method based on the improved YOLOv8 algorithm.

[0071] In summary, a pipeline defect detection method and device based on an improved YOLOv8 algorithm according to an embodiment of the present invention has the following advantages: 1. The embodiment of the present invention replaces the feature extraction network of YOLOv8 with MobileNetV4 to reduce the number of parameters and computational overhead of the model, so that it can run efficiently on embedded devices or mobile devices, thereby realizing real-time pipeline defect detection.

[0072] 2. The embodiment of the present invention introduces an adaptive image enhancement module based on CPA-Enhancer, which can automatically enhance degraded images during the detection process through a chain thinking prompt mechanism, and can dynamically adapt to different image degradation conditions. Unlike traditional fixed enhancement methods, CPA-Enhancer does not need to know the type of image degradation in advance, and has high flexibility and robustness. Especially when processing pipeline inspection videos, it can effectively deal with image quality degradation caused by shaking of the shooting equipment, changes in lighting conditions, etc., thereby improving the detection accuracy and stability of the model in complex environments.

[0073] 3. In order to solve the inconsistency problem between classification and positioning tasks in target detection, FocalLoss is often used for optimization in the past. Focal Loss introduces a modulation factor to reduce the contribution of easy-to-classify samples to the loss, so that the model can pay more attention to those difficult-to-classify samples. However, although Focal Loss has achieved good results in some scenarios, it still has limitations in more complex tasks. The embodiment of the present invention adopts an improved Quality Focal Loss (QFL) to directly integrate the positioning quality of the target (such as IoU score) into the classification loss, thereby effectively solving the inconsistency problem between classification and positioning tasks and improving the comprehensive detection performance of the model.

[0074] In some selectable embodiments, the function / operation mentioned in the block diagram may not occur in the order mentioned in the operation diagram. For example, depending on the function / operation involved, the two boxes shown in succession can actually be executed substantially simultaneously or the boxes can sometimes be executed in reverse order. In addition, the embodiment presented and described in the flow chart of the present invention is provided by way of example, for the purpose of providing a more comprehensive understanding of technology. The disclosed method is not limited to the operation and logic flow presented herein. Selectable embodiments are expected, wherein the order of various operations is changed and the sub-operation of a part for which is described as a larger operation is performed independently.

[0075] In addition, although the present invention is described in the context of functional modules, it should be understood that, unless otherwise specified, one or more of the functions and / or features described may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in separate physical devices or software modules. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the present invention. More specifically, in view of the properties, functions, and internal relationships of the various functional modules in the device disclosed herein, the actual implementation of the module will be understood within the conventional skills of the engineer. Therefore, those skilled in the art can implement the present invention set forth in the claims without excessive experimentation using ordinary techniques. It is also understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of the present invention, which is determined by the full scope of the appended claims and their equivalents.

[0076] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0077] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0078] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0079] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0080] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0081] Although the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

[0082] The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the described embodiments. Those skilled in the art may make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of the present invention.

Claims

1. A pipeline defect detection method based on an improved YOLOv8 algorithm, characterized in that: The following steps are involved: Analyze the pipe segment video to obtain the initial pipe segment image; By annotating the initial pipe section image, a pipe network defect data set is constructed; Performing enhancement processing on the initial pipe section image to obtain a target pipe section image; The initial network structure of the initial pipeline defect detection model is updated, and a target loss function is constructed to obtain a target pipeline defect detection model; Training the target pipeline defect detection model according to the pipeline network defect data set; A detection result is obtained based on the trained target pipeline defect detection model and the target pipe section image.

2. The pipeline defect detection method based on the improved YOLOv8 algorithm according to claim 1, characterized in that: Before the step of parsing the pipe segment video to obtain the initial pipe segment image, the following steps are also included: Reading the pipe network assessment report, extracting information from the pipe network assessment report, and obtaining the pipe section name; According to the pipe segment name, the corresponding pipe segment video is searched and collected.

3. The pipeline defect detection method based on the improved YOLOv8 algorithm according to claim 1, characterized in that: The process of constructing a pipeline network defect dataset by annotating the initial pipeline segment image includes the following steps: Identifying defect locations and defect types in the initial pipe segment image; Annotate the initial pipe segment image according to the defect position and the defect type to obtain an annotation result; According to the labeling results, the pipeline network defect dataset is constructed.

4. The pipeline defect detection method based on the improved YOLOv8 algorithm according to claim 1 is characterized in that: The step of performing enhancement processing on the initial pipe segment image to obtain a target pipe segment image comprises the following steps: Inputting the initial pipe segment image into a CPA-Enhancer enhancement module to generate a target prompt for the initial pipe segment image; Obtaining input features of the initial pipe segment image; The input feature is interacted with the target prompt to obtain the target pipe segment image.

5. The pipeline defect detection method based on the improved YOLOv8 algorithm according to claim 4 is characterized in that: The step of inputting the initial pipe segment image into the CPA-Enhancer enhancement module to generate a target prompt for the initial pipe segment image includes the following steps: Input the initial pipe segment image into the CPA-Enhancer enhancement module, and generate an initial prompt after passing through the first decoding layer; A deconvolution operation is performed on the initial prompt to obtain the target prompt.

6. The pipeline defect detection method based on the improved YOLOv8 algorithm according to claim 4 is characterized in that: The step of interacting the input feature with the target prompt to obtain the target pipe segment image includes the following steps: Obtaining channel weights and spatial weights of the input features; The input feature, the channel weight and the spatial weight are fused to obtain an enhanced feature; The enhanced feature is interactively processed with the target prompt to obtain the target pipe segment image.

7. The pipeline defect detection method based on the improved YOLOv8 algorithm according to claim 1 is characterized in that: The updating of the initial network structure of the initial pipeline defect detection model comprises the following steps: Get the universal inverted bottleneck module, neural architecture search technology, and mobile multi-query attention mechanism; According to the universal inverted bottleneck module, the neural architecture search technology and the mobile version of the multi-query attention mechanism, the initial network structure is updated to obtain a target network structure.

8. The pipeline defect detection method based on the improved YOLOv8 algorithm according to claim 1 is characterized in that: The construction of the target loss function comprises the following steps: Preset quality labels and non-negative parameters; Obtaining an absolute distance between the predicted value and the quality label; Obtaining an adjustment factor according to the absolute distance and the non-negative parameter; The target loss function is obtained according to the adjustment factor, the quality label and the predicted value.

9. A pipeline defect detection device based on an improved YOLOv8 algorithm, characterized in that: include: The first module is used to parse the pipe segment video to obtain the initial pipe segment image; The second module is used to construct a pipeline network defect data set by annotating the initial pipeline segment image; The third module is used to enhance the initial pipe section image to obtain a target pipe section image; The fourth module is used to update the initial network structure of the initial pipeline defect detection model and construct a target loss function to obtain a target pipeline defect detection model; A fifth module is used to train the target pipeline defect detection model according to the pipeline network defect data set; The sixth module is used to obtain the detection result according to the trained target pipeline defect detection model and the target pipe section image.

10. An electronic device, characterized in that: including a processor and a memory; The memory is used to store programs; The processor executes the program to implement the method according to any one of claims 1 to 8.

Images