Pipeline defect detection method and apparatus based on polymorphic rectangular feature extraction network, and electronic device

Abstract

The present application relates to the technical field of oil and gas pipeline inspection. Provided are a pipeline defect detection method and apparatus based on a polymorphic rectangular feature extraction network, and an electronic device. In the method provided in the present application, horizontal polymorphic rectangular pooling processing, vertical polymorphic rectangular pooling processing and elongated convolution processing are performed on sample images, so as to obtain a plurality of feature maps with enhanced long-range inter-pixel dependencies that correspond to the sample images, and each of the plurality of feature maps has both long-distance information and short-distance information. In this way, a neural network model trained on the basis of the plurality of feature maps of each sample image can accurately detect square defects and elongated defects inside a pipeline.

Description

Pipeline defect detection method, device and electronic equipment based on polymorphic rectangular feature extraction network

[0001] This application claims priority to Chinese Patent Application No. 202411215631.5, filed with the Chinese Patent Office on September 2, 2024, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of oil and gas pipeline inspection technology, and for example to a pipeline defect detection method, device and electronic equipment based on a polymorphic rectangular feature extraction network. Background Technology

[0003] Oil and gas pipelines can efficiently and safely transport large quantities of oil and gas resources, ensuring a normal supply. However, with the continuous expansion of oil and gas pipeline networks, pipeline safety issues are becoming increasingly prominent. When oil and gas pipeline safety accidents occur, they often result in casualties and property damage. Therefore, in order to improve the reliability and safety of pipeline operation, pre-commissioning inspections are usually required before pipelines are put into operation to identify any defects.

[0004] In related technologies, pre-construction pipeline inspection typically uses robotic arms to detect deformation defects, but this method cannot detect internal cracks. To effectively detect cracks, a detector equipped with a high-definition camera is usually used to photograph the inside of the pipeline. Machine learning methods are then used to analyze the captured images through square convolution or pooling operations to understand the internal condition of the pipeline. However, this detection method suffers from poor accuracy, failing to accurately identify cracks in near-rectangular areas inside the pipeline, thus failing to meet user needs. Summary of the Invention

[0005] This application provides a pipeline defect detection method based on a polymorphic rectangular feature extraction network, which can quickly and accurately detect cracks and defects inside pipelines, thereby improving detection accuracy and effectively reducing detection time and improving detection efficiency.

[0006] The embodiments of this application adopt the following technical solutions:

[0007] Firstly, a pipeline defect detection method based on a polymorphic rectangular feature extraction network is provided. The method includes: acquiring a training sample set, which includes multiple sample images and the defect detection result corresponding to each sample image; the sample images are acquired by an image sensor inside the pipeline; inputting the multiple sample images into a feature pyramid network, outputting feature maps C1, C2, and C3 for each sample image, wherein the image sizes of feature maps C1, C2, and C3 for each sample image are different; performing horizontal polymorphic rectangular pooling and vertical polymorphic rectangular pooling on feature maps C1, C2, and C3 for each sample image to obtain feature map C of each sample image. The feature maps C11, C21, and C31 corresponding to feature map C1, C2, and C31 of each sample image are processed by long-narrow convolution to obtain feature maps C12, C22, and C32 corresponding to feature map C1 and C21 of each sample image. Specifically, feature maps C11, C21, and C31 of each sample image are feature maps including long-range contextual information, while feature maps C12, C22, and C32 of each sample image are feature maps with long-range spatial dependency enhancement. The feature maps C11 and C21 of each sample image are then processed by long-narrow convolution to obtain feature maps C12, C22, and C32 corresponding to feature map C1 and C21 of each sample image. Figure C12 is overlaid to obtain feature map P1 for each sample image. Feature maps C21 and C22 of each sample image are overlaid to obtain feature map P2 for each sample image. Feature maps C31 and C32 of each sample image are overlaid to obtain feature map P3. Feature map P3 of each sample image is convolved to make the number of channels of feature map P3 of each sample image reach a first preset value to obtain feature map Q3 of each sample image. Feature map P3 of each sample image is sampled and enlarged and then overlaid with feature map P2 of each sample image to obtain the first feature map of each sample image. Convolution processing is performed to make the number of channels in the first feature map of each sample image reach a first preset value, resulting in feature map Q2 for each sample image. Feature map P2 of each sample image is then sampled and enlarged, and then superimposed with feature map P1 of each sample image to obtain the second feature map of each sample image. Convolution processing is performed on the second feature map of each sample image to make the number of channels in the second feature map of each sample image reach the first preset value, resulting in feature map Q1 for each sample image. Convolution processing is performed on feature map Q3 of each sample image to obtain feature map Q4 for each sample image. Convolution processing is performed on feature map Q4 of each sample image to obtain feature map Q5 for each sample image.The neural network model is trained using feature maps Q1, Q2, Q3, Q4, and Q5 of each sample image and the defect detection results, resulting in a trained neural network model. The internal image of the pipe to be detected is input into the trained neural network model, which outputs the defect detection results for the corresponding pipe. These results include whether the defect detection is normal or abnormal.

[0008] In one possible implementation of the first aspect, horizontal polymorphic rectangular pooling and vertical polymorphic rectangular pooling are performed on feature maps C1, C2, and C3 of each sample image to obtain feature map C11 corresponding to feature map C1, feature map C21 corresponding to feature map C2, and feature map C31 corresponding to feature map C3 of each sample image. This includes: performing horizontal polymorphic rectangular pooling on feature maps C1, C2, and C3 of each sample image to obtain feature map I11 corresponding to feature map C1, feature map I21 corresponding to feature map C2, and feature map I31 corresponding to feature map C3 of each sample image, wherein feature maps I11, I21, and I31 are feature maps including horizontal pixel correlation information; and performing horizontal polymorphic rectangular pooling on feature maps C1, C2, and C3 of each sample image. Vertical polymorphic rectangular pooling is used to obtain feature map I12, feature map I22, and feature map I32 corresponding to feature map C1, C2, and C3 of each sample image. Feature maps I12, I22, and I32 are feature maps that include vertical pixel correlation information. Channel stacking and channel weighting are performed on feature maps I11 and I12 of each sample image to obtain feature map C11 corresponding to feature map C1. Channel stacking and channel weighting are performed on feature maps I21 and I22 of each sample image to obtain feature map C21 corresponding to feature map C2. Channel stacking and channel weighting are performed on feature maps I31 and I32 of each sample image to obtain feature map C31 corresponding to feature map C3.

[0009] In one possible implementation of the first aspect, horizontal polymorphic rectangular pooling is performed on feature maps C1, C2, and C3 of each sample image to obtain feature map I11 corresponding to feature map C1, feature map I21 corresponding to feature map C2, and feature map I31 corresponding to feature map C3 of each sample image. This includes: pooling feature maps C1, C2, and C3 of each sample image using multiple horizontal pooling kernels of different sizes to obtain multiple sub-feature maps corresponding to feature map C1, feature map C2, and feature map C3 of each sample image; expanding the multiple sub-feature maps corresponding to feature map C1 of each sample image to obtain multiple sub-feature maps with the same image size as feature map C1; and expanding the multiple sub-feature maps corresponding to feature map C1 of each sample image to obtain multiple sub-feature maps with the same image size as feature map C1. For each sample image, feature maps of the same size are stacked and restored through channel stacking to obtain feature map I11 corresponding to feature map C1. Multiple sub-feature maps corresponding to feature map C2 of each sample image are then expanded to obtain multiple sub-feature maps of the same image size as feature map C2. Channel stacking and channel restoration are then performed on these multiple sub-feature maps of the same image size as feature map C2 to obtain feature map I21 corresponding to feature map C2 of each sample image. Similarly, multiple sub-feature maps corresponding to feature map C3 of each sample image are then expanded to obtain multiple sub-feature maps of the same image size as feature map C3. Finally, channel stacking and channel restoration are performed on these multiple sub-feature maps of the same image size as feature map C3 of each sample image to obtain feature map I31 corresponding to feature map C3 of each sample image.

[0010] In one possible implementation of the first aspect, vertical polymorphic rectangular pooling is performed on feature maps C1, C2, and C3 of each sample image to obtain feature map I12, feature map I22, and feature map I32 corresponding to feature map C1, C2, and C3 of each sample image. This includes: pooling feature maps C1, C2, and C3 of each sample image using multiple vertical pooling kernels of different sizes to obtain multiple sub-feature maps corresponding to feature map C1, C2, and C3 of each sample image; expanding the multiple sub-feature maps corresponding to feature map C1 of each sample image to obtain multiple sub-feature maps with the same image size as feature map C1; and expanding the multiple sub-feature maps corresponding to feature map C1 of each sample image to obtain multiple sub-feature maps with the same image size as feature map C1. For each sample image, feature maps of the same size are stacked and restored through channel stacking to obtain feature map I12 corresponding to feature map C1. Multiple sub-feature maps corresponding to feature map C2 of each sample image are then expanded to obtain multiple sub-feature maps of the same image size as feature map C2. Channel stacking and channel restoration are then performed on these multiple sub-feature maps of the same image size as feature map C2 to obtain feature map I22 corresponding to feature map C2 of each sample image. Similarly, multiple sub-feature maps corresponding to feature map C3 of each sample image are then expanded to obtain multiple sub-feature maps of the same image size as feature map C3. Finally, channel stacking and channel restoration are performed on these multiple sub-feature maps of the same image size as feature map C3 of each sample image to obtain feature map I32 corresponding to feature map C3 of each sample image.

[0011] In one possible implementation of the first aspect, long-narrow convolution processing is performed on feature maps C1, C2, and C3 of each sample image to obtain feature map C12, feature map C22, and feature map C32 corresponding to feature map C1, feature map C22, and feature map C32 corresponding to feature map C3 of each sample image. This includes: performing convolution processing on feature maps C1, C2, and C3 of each sample image using multiple rectangular convolution kernels of different sizes to obtain multiple sub-feature maps corresponding to feature map C1, feature map C2, and feature map C32 corresponding to feature map C3 of each sample image. Multiple sub-feature maps corresponding to feature map C3; channel stacking and channel restoration processing are performed on multiple sub-feature maps corresponding to feature map C1 of each sample image to obtain feature map C12 corresponding to feature map C1 of each sample image; channel stacking and channel restoration processing are performed on multiple sub-feature maps corresponding to feature map C2 of each sample image to obtain feature map C22 corresponding to feature map C2 of each sample image; channel stacking and channel restoration processing are performed on multiple sub-feature maps corresponding to feature map C3 of each sample image to obtain feature map C32 corresponding to feature map C3 of each sample image.

[0012] In one possible implementation of the first aspect, the neural network model is trained based on feature maps Q1, Q2, Q3, Q4, and Q5 of each sample image and the defect detection results to obtain a trained neural network model. This includes: receiving model training parameters, which include the number of sample images, the number of training iterations, and the learning rate; and training the neural network model using a target loss function based on the model training parameters, according to the feature maps Q1, Q2, Q3, Q4, and Q5 of each sample image and the defect detection results to obtain a trained neural network model.

[0013] In one possible implementation of the first aspect, before acquiring the training sample set, the method further includes: acquiring multiple sample images and the defect detection result corresponding to each sample image; preprocessing the multiple sample images to obtain multiple preprocessed sample images, wherein the image size of the multiple preprocessed sample images is the target image size; and generating a training sample set based on the multiple preprocessed sample images and the defect detection result corresponding to each sample image.

[0014] In one possible implementation of the first aspect, before preprocessing the multiple sample images, the method further includes: if the number of multiple sample images is less than a preset number of samples, performing image augmentation processing on the multiple sample images so that the number of sample images is greater than or equal to the preset number of samples.

[0015] Secondly, a pipeline defect detection device based on a polymorphic rectangular feature extraction network is provided. The device includes: an acquisition unit configured to acquire a training sample set, which includes multiple sample images and defect detection results corresponding to each sample image; the sample images are acquired by an image sensor inside the pipeline; a processing unit configured to input the multiple sample images into a feature pyramid network and output feature maps C1, C2, and C3 for each sample image, wherein the image sizes of feature maps C1, C2, and C3 for each sample image are different; the processing unit is further configured to perform horizontal polymorphic rectangular pooling and vertical polymorphic rectangular pooling processing on feature maps C1, C2, and C3 of each sample image. The processing unit obtains feature maps C11, C21, and C31 corresponding to feature map C1, C2, and C31 for each sample image. It also performs long-narrow convolution on feature maps C1, C2, and C3 of each sample image to obtain feature maps C12, C22, and C32 corresponding to feature map C1 and C2, respectively. Feature maps C11, C21, and C31 of each sample image are feature maps including long-range contextual information, while feature maps C12, C22, and C32 of each sample image are feature maps with long-range spatial dependency enhancement. The processing unit is further configured to process each sample image... The feature maps C11 and C12 of this image are superimposed to obtain feature map P1 for each sample image. Feature maps C21 and C22 of each sample image are superimposed to obtain feature map P2 for each sample image. Feature maps C31 and C32 of each sample image are superimposed to obtain feature map P3. The processing unit is further configured to perform convolution processing on feature map P3 of each sample image, so that the number of channels of feature map P3 of each sample image is a first preset value, to obtain feature map Q3 of each sample image. Feature map P3 of each sample image is then sampled and enlarged, and superimposed with feature map P2 of each sample image to obtain the first feature map of each sample image. The first feature map of each sample image is convolved to make the number of channels of the first feature map of each sample image reach a first preset value, resulting in feature map Q2 of each sample image. The feature map P2 of each sample image is sampled and enlarged, and then superimposed with the feature map P1 of each sample image to obtain the second feature map of each sample image. The second feature map of each sample image is convolved to make the number of channels of the second feature map of each sample image reach the first preset value, resulting in feature map Q1 of each sample image. The feature map Q3 of each sample image is convolved to obtain feature map Q4 of each sample image. The feature map Q4 of each sample image is convolved to obtain feature map Q5 of each sample image.The training unit is configured to train the neural network model based on feature maps Q1, Q2, Q3, Q4, and Q5 of each sample image and the defect detection results, resulting in a trained neural network model. The detection unit is configured to input an image of the interior of the pipe to be detected into the trained neural network model and output the defect detection results for the corresponding pipe. The defect detection results include whether the detection is normal or abnormal.

[0016] Thirdly, an electronic device is provided, comprising a memory and at least one processor; the memory is coupled to the processor; wherein the memory stores computer program code, the computer program code including computer instructions, and when the computer instructions are executed by the processor, the electronic device performs the pipeline defect detection method based on a polymorphic rectangular feature extraction network as in any implementation of the first aspect.

[0017] Fourthly, a computer-readable storage medium is provided, including computer instructions that, when executed on an electronic device, cause the electronic device to perform a pipeline defect detection method based on a polymorphic rectangular feature extraction network as described in any implementation of the first aspect.

[0018] Fifthly, a computer program product is provided that, when run on a computer, causes the computer to execute the pipeline defect detection method based on a polymorphic rectangular feature extraction network as described in any implementation of the first aspect. Attached Figure Description

[0019] Figure 1 is a schematic diagram of the hardware structure of an electronic device according to an embodiment of this application;

[0020] Figure 2 is a flowchart of a pipeline defect detection method based on a polymorphic rectangular feature extraction network provided in an embodiment of this application;

[0021] Figure 3 is a flowchart of another pipeline defect detection method based on a polymorphic rectangular feature extraction network provided in an embodiment of this application;

[0022] Figure 4 is a schematic diagram illustrating a process for determining feature map I11 according to an embodiment of this application;

[0023] Figure 5 is a schematic diagram illustrating a process for determining feature map I12 according to an embodiment of this application;

[0024] Figure 6 is a schematic diagram illustrating a process for determining feature map C12 according to an embodiment of this application;

[0025] Figure 7 is a schematic diagram of the hardware structure of a defect detection device according to an embodiment of this application. Detailed Implementation

[0026] The technical solutions of the embodiments of this application will be described below with reference to the accompanying drawings. In the description of this application, unless otherwise stated, " / " indicates that the objects before and after are in an "or" relationship. For example, A / B can represent A or B. "And / or" in this application is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone, where A and B can be singular or plural. Furthermore, in the description of this application, unless otherwise stated, "multiple" refers to two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0027] Furthermore, to facilitate a clear description of the technical solutions in the embodiments of this application, the terms "first" and "second" are used in the embodiments of this application to distinguish identical or similar items with substantially the same function and effect. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and the terms "first" and "second" are not necessarily different.

[0028] In this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being better or more advantageous than other embodiments or designs. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner to facilitate understanding.

[0029] Oil and gas pipelines can efficiently and safely transport large quantities of oil and gas resources, ensuring a normal supply. However, with the continuous expansion of oil and gas pipeline networks, pipeline safety issues are becoming increasingly prominent. When oil and gas pipeline safety accidents occur, they often result in casualties and property damage. Therefore, in order to improve the reliability and safety of pipeline operation, pre-commissioning inspections are usually required before pipelines are put into operation to identify any defects.

[0030] In related technologies, pre-construction pipeline inspection typically uses robotic arms to detect deformation defects, but this method cannot detect internal cracks. To effectively detect cracks, a detector equipped with a high-definition camera is usually used to photograph the inside of the pipeline. Machine learning methods are then used to analyze the captured images through square convolution or pooling operations to understand the internal condition of the pipeline. However, this detection method suffers from poor accuracy, failing to accurately identify cracks in near-rectangular areas inside the pipeline, thus failing to meet user needs.

[0031] In view of this, embodiments of this application provide a pipeline defect detection method based on a polymorphic rectangular feature extraction network. The method includes: acquiring a training sample set, which includes multiple sample images and the defect detection result corresponding to each sample image; the sample images are acquired by an image sensor inside the pipeline; inputting the multiple sample images into a feature pyramid network, outputting feature maps C1, C2, and C3 for each sample image, wherein the image sizes of feature maps C1, C2, and C3 for each sample image are different; performing horizontal polymorphic rectangular pooling and vertical polymorphic rectangular pooling on feature maps C1, C2, and C3 of each sample image to obtain the feature map of each sample image. Feature maps C11 (corresponding to feature map C1), C21 (corresponding to feature map C2), and C31 (corresponding to feature map C3) for each sample image are processed by long-narrow convolution to obtain feature maps C12 (corresponding to feature map C1), C22 (corresponding to feature map C2), and C32 (corresponding to feature map C3) for each sample image. Feature maps C11, C21, and C31 for each sample image are feature maps including long-range contextual information, while feature maps C12, C22, and C32 for each sample image are feature maps with enhanced long-range spatial dependencies. The feature map C11 of each sample image is then processed by long-narrow convolution. The feature maps C11 and C22 of each sample image are superimposed to obtain feature map P1. The feature maps C21 and C22 of each sample image are superimposed to obtain feature map P2. The feature maps C31 and C32 of each sample image are superimposed to obtain feature map P3. The feature map P3 of each sample image is then convolved to ensure that the number of channels in feature map P3 is at a first preset value, resulting in feature map Q3. The feature map P3 of each sample image is then sampled and enlarged before being superimposed with feature map P2 to obtain the first feature map of each sample image. The image is convolved to make the number of channels in the first feature map of each sample image reach a first preset value, resulting in feature map Q2 for each sample image. Feature map P2 of each sample image is then sampled and enlarged, and then superimposed with feature map P1 of each sample image to obtain the second feature map of each sample image. The second feature map of each sample image is convolved to make the number of channels in the second feature map of each sample image reach the first preset value, resulting in feature map Q1 for each sample image. Feature map Q3 of each sample image is convolved to obtain feature map Q4 for each sample image. Feature map Q4 of each sample image is convolved to obtain feature map Q5 for each sample image.The neural network model is trained using feature maps Q1, Q2, Q3, Q4, and Q5 of each sample image and the defect detection results, resulting in a trained neural network model. The internal image of the pipe to be detected is input into the trained neural network model, which outputs the defect detection results for the corresponding pipe. These results include whether the defect detection is normal or abnormal.

[0032] The method provided in this application performs horizontal polymorphic rectangular pooling, vertical polymorphic rectangular pooling, and long-narrow convolution processing on sample images to obtain multiple feature maps that enhance the long-range dependency between pixels in the sample images. Each feature map contains both long-distance and short-distance information. In this way, the neural network model trained based on the multiple feature maps of each sample image can accurately detect square defects and long-narrow defects inside the pipe, thereby improving the detection accuracy. Furthermore, the method provided in this application can effectively reduce detection time and improve detection efficiency.

[0033] In some embodiments, the method provided in this application can be executed by a pipeline defect detection device 100 based on a polymorphic rectangular feature extraction network (hereinafter referred to as defect detection device 100). As an example, defect detection device 100 can be any electronic device 200 with data processing capabilities, such as a general-purpose computer, personal computer, laptop computer, switch, or tablet computer, etc. The specific implementation of defect detection device 100 is not limited here.

[0034] Figure 1 shows a schematic diagram of the hardware structure of an electronic device provided in an embodiment of this application. The electronic device 200 includes a processor 210, a memory 220, and a communication interface 230.

[0035] Processor 210 may include at least one processing core. Processor 210 connects to various parts within electronic device 200 using various interfaces and lines, and performs various functions and processes data of electronic device 200 by running or executing instructions, programs, code sets, or instruction sets stored in memory 220, and by calling data stored in memory 220. Optionally, processor 210 may be implemented using at least one of the following hardware forms: Central Processing Unit (CPU), Graphics Processing Unit (GPU), Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), and Programmable Logic Array (PLA).

[0036] The memory 220 may include random access memory (RAM) or read-only memory (ROM). Optionally, the memory 220 may include a non-transitory computer-readable storage medium. The memory 220 may be configured to store instructions, programs, code, code sets, or instruction sets. The memory 220 may include a stored program area. This stored program area may store instructions for implementing an operating system, instructions for implementing at least one function (such as image acquisition, image processing, and model training functions), and instructions for implementing the various method embodiments described above.

[0037] The communication interface 230 is configured to communicate with other devices, equipment, or communication networks, such as data storage devices, image processing devices, or Ethernet, Radio Access Network (RAN), Wireless Local Area Network (WLAN), etc.

[0038] In terms of physical implementation, the aforementioned devices (such as processor 210, memory 220, and communication interface 230) can each be devices within the same device (such as a laptop computer). Alternatively, at least two of these devices can be located within the same device, i.e., as different devices within the same device, similar to the deployment of devices or components in a distributed system.

[0039] It is understood that the structure illustrated in this embodiment does not constitute a specific limitation on the electronic device 200. In other embodiments of this application, the electronic device 200 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0040] The following description, in conjunction with the accompanying drawings, illustrates the pipeline defect detection method based on a polymorphic rectangular feature extraction network provided in this application.

[0041] Figure 2 is a flowchart of a pipeline defect detection method based on a polymorphic rectangular feature extraction network provided in an embodiment of this application. Optionally, this method can be executed by an electronic device 200 having the hardware structure shown in Figure 1, that is, by a defect detection device 100. The method may include the following steps S1-S8.

[0042] S1. Obtain the training sample set, which includes multiple sample images and the defect detection results corresponding to each sample image.

[0043] For example, the sample images are acquired by the image sensor inside the pipe. The defect detection results corresponding to each sample image include abnormal detection and normal detection. The sample image with the defect detection result of abnormal detection indicates that there is a defect inside the pipe, and the sample image with the defect detection result of normal detection indicates that there is no defect inside the pipe.

[0044] In one possible implementation, prior to S1 above, the method provided in this application embodiment further includes the following steps: acquiring multiple sample images and defect detection results corresponding to each sample image; preprocessing the multiple sample images to obtain multiple preprocessed sample images, wherein the image size of the multiple preprocessed sample images is the target image size; and generating a training sample set based on the multiple preprocessed sample images and the defect detection results corresponding to each sample image.

[0045] The target image size is 640*640.

[0046] In some embodiments, before preprocessing the multiple sample images, the method provided in this application further includes the following steps: when the number of multiple sample images is less than a preset number of samples, performing image augmentation processing on the multiple sample images so that the number of sample images is greater than or equal to the preset number of samples.

[0047] For example, image augmentation processing includes operations such as rotating, scaling, cropping, and flipping the sample images to increase the number of sample images.

[0048] For example, the number of sample images is 100, and the preset number of samples is 200. The defect detection device 100 performs operations such as rotation, scaling, cropping and flipping on the 100 sample images to obtain 100 new sample images. Then, a training sample set is generated based on the 200 sample images, where the image size of the 200 sample images is 640*640.

[0049] The method provided in this application embodiment can effectively increase the number of sample images by augmenting the sample images, thereby avoiding the problem of overfitting of the neural network model due to insufficient sample images, and effectively improving the accuracy of the neural network model in defect detection.

[0050] S2. Input multiple sample images into the feature pyramid network and output feature maps C1, C2 and C3 for each sample image.

[0051] Among them, the feature maps C1, C2, and C3 of each sample image have different image sizes.

[0052] Feature Pyramid Networks (FPNs) are deep neural networks used for object detection and semantic segmentation. Their core idea is to construct a feature pyramid using cross-layer connections and a top-level feature pyramid to preserve the semantic information of high-level features and the spatial information of low-level features. By constructing feature pyramids at different scales and extracting object features at different scales, FPNs improve the accuracy of detection and segmentation.

[0053] Specifically, feature maps C1, C2, and C3 are obtained for each of the multiple sample images through the feature pyramid network. Feature maps C1, C2, and C3 can include a small range of contextual information in the sample images. The neural network model trained with this small range of contextual information can remain sensitive to pit and pinhole defects inside the pipe.

[0054] In one example, the image sizes of feature map C1, feature map C2, and feature map C3 are 80×80×512, 40×40×1024, and 20×20×2048, respectively. This can also be understood as feature map C1 having a width and height of 80 and 80, with 512 channels; feature map C2 having a width and height of 40 and 40, with 1024 channels; and feature map C3 having a width and height of 20 and 20, with 2048 channels.

[0055] S3. Perform horizontal polymorphic rectangular pooling and vertical polymorphic rectangular pooling on feature maps C1, C2 and C3 of each sample image to obtain feature map C11 corresponding to feature map C1, feature map C21 corresponding to feature map C2 and feature map C31 corresponding to feature map C3 of each sample image.

[0056] Among them, feature maps C11, C21 and C31 of each sample image are feature maps that include long-range contextual information, and feature maps C12, C22 and C32 of each sample image are feature maps with long-range spatial dependency enhancement.

[0057] In some embodiments, referring to FIG3, the above S3 includes the following steps S21-S23.

[0058] S21. Perform horizontal polymorphic rectangular pooling on feature maps C1, C2 and C3 of each sample image to obtain feature map I11 corresponding to feature map C1, feature map I21 corresponding to feature map C2 and feature map I31 corresponding to feature map C3 of each sample image.

[0059] Among them, feature map I11, feature map I21 and feature map I31 are feature maps that include horizontal pixel correlation information.

[0060] In one possible implementation, lateral polymorphic rectangular pooling is performed on feature maps C1, C2, and C3 of each sample image to obtain feature map I11 corresponding to feature map C1, feature map I21 corresponding to feature map C2, and feature map I31 corresponding to feature map C3 of each sample image, including:

[0061] The feature maps C1, C2, and C3 of each sample image are pooled using multiple horizontal pooling kernels of different sizes to obtain multiple sub-feature maps corresponding to feature map C1, feature map C2, and feature map C3 of each sample image.

[0062] The feature maps corresponding to feature map C1 of each sample image are expanded to obtain multiple sub-feature maps with the same image size as feature map C1. Channel stacking and channel restoration processing are performed on the multiple sub-feature maps with the same image size as feature map C1 of each sample image to obtain feature map I11 corresponding to feature map C1 of each sample image. The feature maps corresponding to feature map C2 of each sample image are expanded to obtain multiple sub-feature maps with the same image size as feature map C2. Channel stacking and channel restoration processing are performed on the multiple sub-feature maps with the same image size as feature map C2 of each sample image to obtain feature map I21 corresponding to feature map C2 of each sample image. The feature maps corresponding to feature map C3 of each sample image are expanded to obtain multiple sub-feature maps with the same image size as feature map C3. Channel stacking and channel restoration processing are performed on the multiple sub-feature maps with the same image size as feature map C3 of each sample image to obtain feature map I31 corresponding to feature map C3 of each sample image.

[0063] For example, the defect detection device 100 performs lateral polymorphic rectangular pooling processing on feature maps C1, C2, and C3 of each sample image through the lateral polymorphic rectangular pooling module in the polymorphic rectangular pooling module. The lateral polymorphic rectangular pooling module includes multiple lateral pooling kernels of different sizes. These kernels are used to collect information from different lateral regions of feature maps C1, C2, and C3, that is, the lateral pixel correlation information of feature maps C1, C2, and C3. Referring to Figure 4, when the image size of feature map C1 is h×w, where h is the height of the feature map and w is the width of the feature map, respectively... Horizontal pooling kernels of varying sizes are used to pool the feature maps, resulting in multiple image sizes. The sub-feature maps are then expanded using nearest neighbor upsampling, resulting in image sizes of h×w for all sub-feature maps. After channel stacking, the information from each lateral region is integrated, and finally, a 1×1 convolution is used to restore the channels to c (where c is the number of channels in feature map C1), ensuring that the resulting feature map I11 has the same image size as feature map C1. It should be noted that the process of obtaining feature map I21 from feature map C2 is the same as the process of obtaining I11 from feature map C1, and the process of obtaining feature map I31 from feature map C3 is the same as the process of obtaining I11 from feature map C1; these will not be elaborated further here.

[0064] The generation process of feature map I11, feature map I21, and feature map I31 can be as follows:

[0065] Among them, I W It can be any one of feature map I11, feature map I21, and feature map I31.

[0066] S22. Perform vertical polymorphic rectangular pooling on feature maps C1, C2 and C3 of each sample image to obtain feature map I12, feature map I22 and feature map I32 corresponding to feature map C1, feature map I22 and feature map I32 corresponding to feature map C3 of each sample image.

[0067] Among them, feature map I12, feature map I22 and feature map I32 are feature maps that include vertical pixel correlation information.

[0068] In one possible implementation, vertical polymorphic rectangular pooling is performed on feature maps C1, C2, and C3 of each sample image to obtain feature map I12 corresponding to feature map C1, feature map I22 corresponding to feature map C2, and feature map I32 corresponding to feature map C3 of each sample image, including:

[0069] The feature maps C1, C2, and C3 of each sample image are pooled using multiple vertical pooling kernels of different sizes to obtain multiple sub-feature maps corresponding to feature map C1, feature map C2, and feature map C3 of each sample image.

[0070] The feature maps corresponding to feature map C1 of each sample image are expanded to obtain multiple sub-feature maps with the same image size as feature map C1. Channel stacking and channel restoration processing are performed on the multiple sub-feature maps with the same image size as feature map C1 of each sample image to obtain feature map I12 corresponding to feature map C1 of each sample image. The feature maps corresponding to feature map C2 of each sample image are expanded to obtain multiple sub-feature maps with the same image size as feature map C2. Channel stacking and channel restoration processing are performed on the multiple sub-feature maps with the same image size as feature map C2 of each sample image to obtain feature map I22 corresponding to feature map C2 of each sample image. The feature maps corresponding to feature map C3 of each sample image are expanded to obtain multiple sub-feature maps with the same image size as feature map C3. Channel stacking and channel restoration processing are performed on the multiple sub-feature maps with the same image size as feature map C3 of each sample image to obtain feature map I32 corresponding to feature map C3 of each sample image.

[0071] For example, referring to Figure 5, the defect detection device 100 performs vertical polymorphic rectangular pooling processing on feature maps C1, C2, and C3 of each sample image through the vertical polymorphic rectangular pooling module in the polymorphic rectangular pooling module. The vertical polymorphic rectangular pooling module includes multiple vertical pooling kernels of different sizes. These kernels are used to collect information from different vertical regions of feature maps C1, C2, and C3, that is, the vertical pixel correlation information of feature maps C1, C2, and C3. When the image size of feature map C1 is h×w, where h is the height of the feature map and w is the width of the feature map, respectively... Pooling is performed using vertical pooling kernels of varying sizes to obtain multiple image sizes. The sub-feature maps are then expanded using nearest neighbor upsampling, resulting in an image size of h×w for each sub-feature map. After channel stacking, the vertical region information is integrated, and finally, a 1×1 convolution is used to restore the channels to c0 (c0 being the number of channels in feature map C2), ensuring that the resulting feature map I12 has the same image size as feature map C1. It should be noted that the process of obtaining feature map I22 from feature map C2 is the same as the process of obtaining I12 from feature map C1, and the process of obtaining feature map I32 from feature map C3 is the same as the process of obtaining I12 from feature map C1; these will not be elaborated further here.

[0072] The generation process of feature map I12, feature map I22, and feature map I32 can be as follows:

[0073] Among them, I h It can be any one of feature map I12, feature map I22, and feature map I32.

[0074] S23. Perform channel stacking and channel weighting on feature maps I11 and I12 of each sample image to obtain feature map C11 corresponding to feature map C1 of each sample image. Perform channel stacking and channel weighting on feature maps I21 and I22 of each sample image to obtain feature map C21 corresponding to feature map C2 of each sample image. Perform channel stacking and channel weighting on feature maps I31 and I32 of each sample image to obtain feature map C31 corresponding to feature map C3 of each sample image.

[0075] For example, feature maps I11 and I12 are stacked by channels, and then the channels are weighted using the eSE (Effective Squeeze-and-Excitation) attention mechanism to reduce the weight of redundant channels, resulting in feature map C11 corresponding to feature map C1. Feature maps I21 and I22 are stacked by channels, and then the channels are weighted using the eSE attention mechanism to reduce the weight of redundant channels, resulting in feature map C21 corresponding to feature map C2. Feature maps I31 and I32 are stacked by channels, and then the channels are weighted using the eSE attention mechanism to reduce the weight of redundant channels, resulting in feature map C31 corresponding to feature map C3.

[0076] The generation process of feature map C11, feature map C21 and feature map C31 can be as follows:

[0077] Among them, I PPM It can be any one of feature map C11, feature map C21 and feature map C31.

[0078] As can be seen from the above, the method provided in this application performs horizontal polymorphic rectangular pooling and vertical polymorphic rectangular pooling on feature maps C11, C21 and C31, thereby enhancing the long-distance correlation between pixels in feature maps C11, C21 and C31. The neural network model trained by feature maps C11, C21 and C31 can accurately and quickly identify crack-type defects.

[0079] S4. Perform long-narrow convolution on feature maps C1, C2, and C3 of each sample image to obtain feature map C12, feature map C22, and feature map C32 corresponding to feature map C1 and feature map C3 of each sample image.

[0080] In one possible implementation, the above S4 includes the following steps:

[0081] The feature maps C1, C2, and C3 of each sample image are convolved using multiple rectangular convolution kernels of different sizes to obtain multiple sub-feature maps corresponding to feature map C1, C2, and C3 of each sample image. Channel stacking and channel restoration are then performed on the multiple sub-feature maps corresponding to feature map C1 of each sample image to obtain feature map C12. Similarly, channel stacking and channel restoration are performed on the multiple sub-feature maps corresponding to feature map C2 of each sample image to obtain feature map C22. Finally, channel stacking and channel restoration are performed on the multiple sub-feature maps corresponding to feature map C3 of each sample image to obtain feature map C32.

[0082] For example, referring to Figure 6, when the image size of feature map C1 is h×w×c, where h is the height of the feature map, w is the width of the feature map, and c is the number of channels of the feature map, the defect detection device 100 performs long-narrow convolution processing on the feature map C1 of each sample image through the Remote Compensation Module. First, the feature map C1 is grouped into 4 groups based on the number of channels of feature map C1, with 64 channels in each group. The grouped feature map C1 is then convolved using rectangular convolution kernels. Multiple rectangular convolution kernels of different sizes include 3 branches, where the rectangular convolution kernel size of branch 1 is 1×5 and 5×1, the rectangular convolution kernel size of branch 2 is 1×10 and 10×1, and the rectangular convolution kernel size of branch 3 is 1×20 and 20×1. Then, the multiple sub-feature maps obtained from multiple rectangular convolution kernels are summed, and then the multiple sub-feature maps are superimposed on each channel. At this point, the image size of feature map C12 corresponding to feature map C1 of each sample image is h×w×(c / 4)×n. Then, the channels are weighted through the eSE channel attention mechanism to increase the weight of the effective channels. Finally, a 1×1 convolution operation is performed to obtain feature map C12 corresponding to feature map C1, and the image size of feature map C12 is h×w×c. It should be understood that the generation method of feature map C22 and feature map C32 is the same as that of feature map C12, and will not be repeated here.

[0083] S5. Overlay feature maps C11 and C12 of each sample image to obtain feature map P1 of each sample image. Overlay feature maps C21 and C22 of each sample image to obtain feature map P2 of each sample image. Overlay feature maps C31 and C32 of each sample image to obtain feature map P3.

[0084] S6. Perform convolution processing on the feature map P3 of each sample image to make the number of channels of the feature map P3 of each sample image reach a first preset value, to obtain the feature map Q3 of each sample image. After sampling and enlarging the feature map P3 of each sample image, superimpose it with the feature map P2 of each sample image to obtain the first feature map of each sample image. Perform convolution processing on the first feature map of each sample image to make the number of channels of the first feature map of each sample image reach a first preset value, to obtain the feature map Q2 of each sample image. After sampling and enlarging the feature map P2 of each sample image, superimpose it with the feature map P1 of each sample image to obtain the second feature map of each sample image. Perform convolution processing on the second feature map of each sample image to make the number of channels of the second feature map of each sample image reach a first preset value, to obtain the feature map Q1 of each sample image. Perform convolution processing on the feature map Q3 of each sample image to obtain the feature map Q4 of each sample image. Perform convolution processing on the feature map Q4 of each sample image to obtain the feature map Q5 of each sample image.

[0085] For example, a 3×3 convolution is performed on feature map Q3 of each sample image to obtain feature map Q4 of each sample image. Feature map Q4 has an image size of 10×10 and a larger receptive field. A 3×3 convolution is then performed on feature map Q4 of each sample image to obtain feature map Q5 of each sample image. Feature map Q5 has an image size of 5×5 and is more sensitive to large defects. Since the convolution in the ResNet network uses conventional square convolution, more local information can be obtained during the initial feature extraction. The superposition of the polymorphic rectangular pooling module and the long-range compensation module helps to improve the long-range dependence between pixels. This allows the polymorphic rectangular feature extraction network to have information on both near and far distances, making it equally sensitive to square defects (such as pinholes, dents, etc.) and long and narrow defects (such as cracks).

[0086] S7. Train the neural network model based on the feature maps Q1, Q2, Q3, Q4, Q5 of each sample image and the defect detection results to obtain the trained neural network model.

[0087] In some embodiments, S7 above includes the following steps:

[0088] Receive model training parameters, including the number of sample images, training iterations, and learning rate; based on the model training parameters, train the neural network model using the target loss function according to the feature maps Q1, Q2, Q3, Q4, and Q5 of each sample image and the defect detection results, to obtain the trained neural network model.

[0089] For example, classification and regression operations are performed on feature maps Q1, Q2, Q3, Q4, and Q5 respectively, and the results are compared with the defect detection results of feature maps Q1, Q2, Q3, Q4, and Q5. Then, the number of sample images is set to 8, the number of training iterations is set to 100, and the learning rate is set to 0.1. After 50 training iterations, the learning rate is set to 0.05. The weights of each layer of the model are modified by calculating the loss function value, and the final trained model weights are saved to obtain the trained neural network model.

[0090] S8. Input the image of the inside of the pipe to be detected into the trained neural network model, and output the defect detection result corresponding to the pipe to be detected. The defect detection result includes normal detection or abnormal detection.

[0091] As can be seen from S1-S8 above, the method provided in this application embodiment obtains multiple feature maps with enhanced long-range dependencies between pixels in the sample image by performing horizontal polymorphic rectangular pooling, vertical polymorphic rectangular pooling, and long narrow convolution processing on the sample image. Each feature map has both long-distance and short-distance information. In this way, the neural network model trained based on the multiple feature maps of each sample image can accurately detect square defects and long narrow defects inside the pipe, thereby improving the detection accuracy. Furthermore, the method provided in this application can effectively reduce detection time and improve detection efficiency.

[0092] The foregoing mainly describes the solutions of the embodiments of this application from a methodological perspective. It is understood that, in order to achieve the above-mentioned functions, the defect detection device 100 includes at least one of the hardware structures and software modules corresponding to the execution of each function. Those skilled in the art should readily recognize that, in conjunction with the units and algorithm steps of the various examples described in the embodiments disclosed herein, the embodiments of this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of this application.

[0093] In this embodiment, the defect detection device 100 can be divided into functional units according to the method example described above. For example, the defect detection device 100 can be divided into functional units corresponding to various functions, or two or more functions can be integrated into one processing unit. The integrated unit can be implemented in hardware or as a software functional unit. It should be noted that the unit division in this embodiment is illustrative and only represents one logical functional division; other division methods may be used in actual implementation.

[0094] For example, Figure 7 shows a schematic diagram of the hardware structure of a defect detection device 100 provided in an embodiment of this application. The defect detection device 100 includes: an acquisition unit 710, configured to acquire a training sample set, the training sample set including multiple sample images and the defect detection result corresponding to each sample image, the sample images being acquired by an image sensor inside a pipeline; a processing unit 720, configured to input multiple sample images into a feature pyramid network, and output feature map C1, feature map C2, and feature map C3 of each sample image, wherein the image sizes of feature map C1, feature map C2, and feature map C3 of each sample image are different; the processing unit 720 is further configured to perform horizontal polymorphic rectangular pooling processing and vertical polymorphic rectangular pooling processing on feature map C1, feature map C2, and feature map C3 of each sample image to obtain each The feature maps C11, C21, and C31 corresponding to feature map C1, C2, and C3 of the sample image are processed by long-narrow convolution to obtain feature maps C12, C22, and C32 corresponding to feature map C1 and C2 of each sample image. Specifically, feature maps C11, C21, and C31 of each sample image are feature maps including long-range contextual information, while feature maps C12, C22, and C32 of each sample image are feature maps with enhanced long-range spatial dependencies. Processing unit 720 The processing unit 720 is further configured to superimpose feature maps C11 and C12 of each sample image to obtain feature map P1, superimpose feature maps C21 and C22 of each sample image to obtain feature map P2, and superimpose feature maps C31 and C32 of each sample image to obtain feature map P3. The processing unit 720 is also configured to perform convolution processing on feature map P3 of each sample image, such that the number of channels in feature map P3 of each sample image is a first preset value, to obtain feature map Q3 of each sample image. After sampling and magnification processing, feature map P3 of each sample image is then combined with feature map P2 of each sample image. The first feature map of each sample image is obtained by overlaying the first feature map. Then, the first feature map of each sample image is convolved to make the number of channels in the first feature map of each sample image reach a first preset value, resulting in feature map Q2. Feature map P2 of each sample image is then sampled and enlarged, and then overlaid with feature map P1 of each sample image to obtain the second feature map of each sample image. The second feature map of each sample image is then convolved to make the number of channels in the second feature map of each sample image reach the first preset value, resulting in feature map Q1. Finally, feature map Q3 of each sample image is convolved to obtain feature map Q4.Convolution is performed on feature map Q4 of each sample image to obtain feature map Q5 of each sample image; training unit 730 is configured to train the neural network model based on feature maps Q1, Q2, Q3, Q4, Q5 of each sample image and the defect detection result to obtain the trained neural network model; detection unit 740 is configured to input the internal image of the pipe to be detected into the trained neural network model and output the defect detection result corresponding to the pipe to be detected, including whether the defect detection is normal or abnormal.

[0095] It should be understood that specific descriptions of the above-mentioned optional methods can be found in the foregoing method embodiments, and will not be repeated here. Furthermore, explanations of any of the defect detection devices 100 provided above, as well as descriptions of their beneficial effects, can be found in the corresponding method embodiments described above, and will not be repeated here.

[0096] This application also provides a computer-readable storage medium storing at least one computer instruction, which is loaded and executed by a processor to implement the methods of the various embodiments described above. Explanations of the relevant content and descriptions of the beneficial effects of any of the computer-readable storage media provided above can be found in the corresponding embodiments described above, and will not be repeated here.

[0097] This application also provides a chip. This chip integrates a control circuit and at least one port for implementing the functions of the defect detection device 100 described above. Optionally, the functions supported by this chip can be referred to above, and will not be repeated here.

[0098] Those skilled in the art will understand that the program for implementing all or part of the steps of the above embodiments, which can be executed by a program instructing related hardware, can be stored in a computer-readable storage medium. The storage medium mentioned above can be a read-only memory, a random access memory, etc. The processing unit or processor mentioned above can be a central processing unit, a general-purpose processor, an application-specific integrated circuit (ASIC), a microprocessor (digital signal processor, DSP), a field-programmable gate array (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof.

[0099] This application also provides a computer program product containing instructions that, when executed on a computer, cause the computer to perform any of the methods described in the above embodiments. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions may be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. Available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., Digital Versatile Discs (DVDs)), or semiconductor media (e.g., Solid State Drives (SSDs)).

[0100] It should be noted that the devices for storing computer instructions or computer programs provided in the embodiments of this application, such as, but not limited to, the aforementioned memory, computer-readable storage medium, and communication chip, are all non-transitory. Those skilled in the art should recognize that the functions described in the embodiments of this application in one or more of the above examples can be implemented using hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable storage medium or transmitted as one or more instructions or code on a computer-readable storage medium. Computer-readable storage media include computer storage media and communication media, wherein communication media include any medium that facilitates the transmission of computer programs from one place to another. Storage media can be any available medium accessible to general-purpose or special-purpose computers.

[0101] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. A pipeline defect detection method based on a polymorphic rectangular feature extraction network, comprising: A training sample set is obtained, which includes multiple sample images and the defect detection results corresponding to each sample image. The sample images are acquired by an image sensor inside the pipe. The multiple sample images are input into the feature pyramid network, and feature maps C1, C2 and C3 of each sample image are output, wherein the image sizes of feature maps C1, C2 and C3 of each sample image are different; For each of the sample images, feature maps C1, C2, and C3 are subjected to horizontal polymorphic rectangular pooling and vertical polymorphic rectangular pooling to obtain feature maps C11, C21, and C31 corresponding to feature map C1, C2, and C3, respectively. Furthermore, for each of the sample images, feature maps C1, C2, and C3 are subjected to long-narrow convolution to obtain feature maps C12, C22, and C32, respectively. Here, feature maps C11, C21, and C31 are feature maps including long-range contextual information, and feature maps C12, C22, and C32 are feature maps with enhanced long-range spatial dependencies. The feature maps C11 and C12 of each sample image are superimposed to obtain the feature map P1 of each sample image; the feature maps C21 and C22 of each sample image are superimposed to obtain the feature map P2 of each sample image; and the feature maps C31 and C32 of each sample image are superimposed to obtain the feature map P3. Each sample image's feature map P3 is convolved to make the number of channels in each sample image's feature map P3 equal to a first preset value, resulting in a feature map Q3 for each sample image. The feature map P3 of each sample image is then sampled and enlarged, and then superimposed with the feature map P2 of each sample image to obtain a first feature map for each sample image. The first feature map of each sample image is then convolved to make the number of channels in each sample image equal to the first preset value, resulting in a feature map Q2 for each sample image. The feature map P2 of each sample image is then sampled and enlarged, and then superimposed with the feature map P1 of each sample image to obtain a second feature map for each sample image. The second feature map of each sample image is then convolved to make the number of channels in each sample image equal to the first preset value, resulting in a feature map Q1 for each sample image. The feature map Q3 of each sample image is then convolved to obtain a feature map Q4 for each sample image. The feature map Q4 of each sample image is then convolved to obtain a feature map Q5 for each sample image. The neural network model is trained based on the feature maps Q1, Q2, Q3, Q4, and Q5 of each sample image and the defect detection results to obtain the trained neural network model. The internal image of the pipe to be detected is input into the trained neural network model, and the defect detection result corresponding to the pipe to be detected is output. The defect detection result includes normal detection or abnormal detection.

2. The method according to claim 1, wherein, The step of performing horizontal and vertical polymorphic rectangular pooling processing on feature maps C1, C2, and C3 of each sample image to obtain feature map C11 corresponding to feature map C1, feature map C21 corresponding to feature map C2, and feature map C31 corresponding to feature map C3 of each sample image includes: Horizontal polymorphic rectangular pooling is performed on feature maps C1, C2, and C3 of each sample image to obtain feature map I11, feature map I21, and feature map I31 corresponding to feature map C1, feature map I21, and feature map I31 corresponding to feature map C3 of each sample image, wherein feature maps I11, I21, and I31 are feature maps that include horizontal pixel correlation information; Vertical polymorphic rectangular pooling is performed on feature maps C1, C2, and C3 of each sample image to obtain feature map I12, feature map I22, and feature map I32 corresponding to feature map C1 and feature map I32 corresponding to feature map C3 of each sample image, wherein feature map I12, feature map I22, and feature map I32 are feature maps that include vertical pixel correlation information; For each sample image, feature maps I11 and I12 are processed by channel stacking and channel weighting to obtain feature map C11 corresponding to feature map C1 of each sample image. For each sample image, feature maps I21 and I22 are processed by channel stacking and channel weighting to obtain feature map C21 corresponding to feature map C2 of each sample image. For each sample image, feature maps I31 and I32 are processed by channel stacking and channel weighting to obtain feature map C31 corresponding to feature map C3 of each sample image.

3. The method according to claim 2, wherein, The step of performing lateral polymorphic rectangular pooling on feature maps C1, C2, and C3 of each sample image to obtain feature map I11 corresponding to feature map C1, feature map I21 corresponding to feature map C2, and feature map I31 corresponding to feature map C3 of each sample image includes: The feature maps C1, C2, and C3 of each sample image are pooled using multiple horizontal pooling kernels of different sizes to obtain multiple sub-feature maps corresponding to feature map C1, feature map C2, and feature map C3 of each sample image. Each sample image's feature map C1 is expanded to obtain multiple sub-feature maps with the same image size as feature map C1. Channel stacking and channel restoration processing are performed on these sub-feature maps to obtain feature map I11 corresponding to feature map C1. Similarly, each sample image's feature map C2 is expanded to obtain multiple sub-feature maps with the same image size as feature map C2. Channel stacking and channel restoration processing are performed on these sub-feature maps to obtain feature map I21 corresponding to feature map C2. Likewise, each sample image's feature map C3 is expanded to obtain multiple sub-feature maps with the same image size as feature map C3. Channel stacking and channel restoration processing are performed on these sub-feature maps to obtain feature map I31 corresponding to feature map C3.

4. The method according to claim 3, wherein, The step of performing vertical polymorphic rectangular pooling on feature maps C1, C2, and C3 of each sample image to obtain feature map I12 corresponding to feature map C1, feature map I22 corresponding to feature map C2, and feature map I32 corresponding to feature map C3 of each sample image includes: The feature maps C1, C2, and C3 of each sample image are pooled using multiple vertical pooling kernels of different sizes to obtain multiple sub-feature maps corresponding to feature map C1, feature map C2, and feature map C3 of each sample image. Each sample image's feature map C1 is expanded to obtain multiple sub-feature maps with the same image size as feature map C1. Channel stacking and channel restoration processing are performed on these sub-feature maps to obtain feature map I12 corresponding to feature map C1. Similarly, each sample image's feature map C2 is expanded to obtain multiple sub-feature maps with the same image size as feature map C2. Channel stacking and channel restoration processing are performed on these sub-feature maps to obtain feature map I22 corresponding to feature map C2. Likewise, each sample image's feature map C3 is expanded to obtain multiple sub-feature maps with the same image size as feature map C3. Channel stacking and channel restoration processing are performed on these sub-feature maps to obtain feature map I32 corresponding to feature map C3.

5. The method according to claim 4, wherein, The step of performing long-narrow convolution processing on feature maps C1, C2, and C3 of each sample image to obtain feature map C12 corresponding to feature map C1, feature map C22 corresponding to feature map C2, and feature map C32 corresponding to feature map C3 of each sample image includes: The feature maps C1, C2, and C3 of each sample image are convolved using multiple rectangular convolution kernels of different sizes to obtain multiple sub-feature maps corresponding to feature map C1, feature map C2, and feature map C3 of each sample image. Multiple sub-feature maps corresponding to feature map C1 of each sample image are subjected to channel stacking and channel restoration processing to obtain feature map C12 corresponding to feature map C1 of each sample image. Multiple sub-feature maps corresponding to feature map C2 of each sample image are subjected to channel stacking and channel restoration processing to obtain feature map C22 corresponding to feature map C2 of each sample image. Multiple sub-feature maps corresponding to feature map C3 of each sample image are subjected to channel stacking and channel restoration processing to obtain feature map C32 corresponding to feature map C3 of each sample image.

6. The method according to claim 5, wherein, The step of training the neural network model based on feature maps Q1, Q2, Q3, Q4, and Q5 of each sample image and the defect detection results to obtain the trained neural network model includes: Receive model training parameters, which include the number of sample images, the number of training iterations, and the learning rate; Based on the model training parameters, the neural network model is trained using the target loss function according to the feature maps Q1, Q2, Q3, Q4, and Q5 of each sample image and the defect detection results, to obtain the trained neural network model.

7. The method according to claim 6, further comprising, before obtaining the training sample set: Acquire multiple sample images and the defect detection results corresponding to each sample image; The multiple sample images are preprocessed to obtain multiple preprocessed sample images, and the image size of the multiple preprocessed sample images is the target image size; The training sample set is generated based on the preprocessed sample images and the defect detection results corresponding to each sample image.

8. The method according to claim 7, wherein before preprocessing the plurality of sample images, the method further comprises: In response to the fact that the number of the plurality of sample images is less than a preset number of samples, image augmentation processing is performed on the plurality of sample images so that the number of sample images is greater than or equal to the preset number of samples.

9. A pipeline defect detection device based on a polymorphic rectangular feature extraction network, comprising: The acquisition unit is configured to acquire a training sample set, which includes multiple sample images and the defect detection results corresponding to each sample image. The sample images are acquired by an image sensor inside the pipeline. The processing unit is configured to input the plurality of sample images into a feature pyramid network and output feature maps C1, C2, and C3 for each sample image, wherein the image sizes of feature maps C1, C2, and C3 for each sample image are different. The processing unit is further configured to perform horizontal polymorphic rectangular pooling and vertical polymorphic rectangular pooling on feature maps C1, C2, and C3 of each sample image to obtain feature map C11, feature map C21, and feature map C31 corresponding to feature map C1, feature map C2, and feature map C31 corresponding to feature map C3 of each sample image; and to perform long narrow convolution on feature maps C1, C2, and C3 of each sample image to obtain feature map C12, feature map C22, and feature map C32 corresponding to feature map C1, feature map C2, and feature map C32 corresponding to feature map C3 of each sample image, wherein feature maps C11, C21, and C31 of each sample image are feature maps including long-range context information, and feature maps C12, C22, and C32 of each sample image are feature maps with long-range spatial dependency enhancement; The processing unit is further configured to superimpose feature maps C11 and C12 of each sample image to obtain feature map P1 of each sample image, superimpose feature maps C21 and C22 of each sample image to obtain feature map P2 of each sample image, and superimpose feature maps C31 and C32 of each sample image to obtain feature map P3. The processing unit is further configured to perform convolution processing on the feature map P3 of each sample image, so that the number of channels of the feature map P3 of each sample image is a first preset value, to obtain a feature map Q3 of each sample image; to perform sampling and magnification processing on the feature map P3 of each sample image and then superimpose it with the feature map P2 of each sample image to obtain a first feature map of each sample image; and to perform convolution processing on the first feature map of each sample image, so that the number of channels of the first feature map of each sample image is the first preset value, to obtain a feature map Q2 of each sample image. The feature map P2 of each sample image is sampled and enlarged, and then superimposed with the feature map P1 of each sample image to obtain the second feature map of each sample image. The second feature map of each sample image is then convolved to make the number of channels in the second feature map of each sample image equal to the first preset value, resulting in feature map Q1 of each sample image. The feature map Q3 of each sample image is then convolved to obtain feature map Q4 of each sample image. Finally, the feature map Q4 of each sample image is convolved to obtain feature map Q5 of each sample image. The training unit is configured to train the neural network model based on the feature maps Q1, Q2, Q3, Q4, and Q5 of each sample image and the defect detection results, thereby obtaining a trained neural network model. The detection unit is configured to input the internal image of the pipe to be detected into the trained neural network model and output the defect detection result corresponding to the pipe to be detected, wherein the defect detection result includes normal detection or abnormal detection.

10. An electronic device, comprising: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the pipeline defect detection method based on a polymorphic rectangular feature extraction network as described in any one of claims 1-8.

Images