A pipeline inner wall defect detection method and system

Abstract

The present invention relates to a pipeline inner wall defect detection method and system, comprising the following steps: acquiring a pipeline inner wall image; determining a defect category feature map and a defect location feature map; determining a target category feature map and a target location feature map based on each defect category feature map and each defect location feature map; determining a regression frame based on each target location feature map; determining classification information and a regression frame feature map based on each regression frame and each target category feature map; determining category information corresponding to the pipeline inner wall image based on each classification information; determining a regression frame score corresponding to each regression frame feature map based on each regression frame feature map; and determining positioning information corresponding to the pipeline inner wall image based on the regression frame score and each classification information. This method solves the problem that existing technologies cannot efficiently display pipeline defect information and defect location in the pipeline to frontline workers, which is not conducive to subsequent pipeline maintenance and repair.

Description

Technical Field

[0001] The present invention relates to the technical field of oil and gas pipeline detection, and in particular to a pipeline inner wall defect detection method and system. Background Art

[0002] Oil and gas pipelines transport the majority of my country's oil and gas resources. As pipelines age, leaks and explosions become common due to factors such as natural disasters, third-party damage, corrosion, and perforations. These incidents pose significant risks to human safety and the ecological environment, necessitating regular pipeline integrity testing. Currently, in-pipeline testing technologies are more mature, typically utilizing magnetic flux leakage testing, eddy current testing, and ultrasonic testing, which can detect defects such as small corrosion pits and cracks.

[0003] However, although the above method can detect defects on the inner wall of the pipeline, it cannot display the defect information in the pipeline and the location of the defect in the pipeline to front-line workers, which is not conducive to subsequent pipeline maintenance and inspection. Summary of the Invention

[0004] In order to overcome the problem that the existing technology cannot display defect information in the pipeline and the location of the defect in the pipeline to front-line workers, which is not conducive to subsequent pipeline maintenance and inspection, the present invention provides a pipeline inner wall defect detection method and system.

[0005] In a first aspect, in order to solve the above technical problems, the present invention provides a method for detecting defects on the inner wall of a pipeline, comprising the following steps:

[0006] Acquire images of the inner wall of the pipeline;

[0007] Determining, based on the pipeline inner wall image, a plurality of defect category feature maps and a plurality of defect position feature maps corresponding to the pipeline inner wall image;

[0008] Determining at least one target category feature map based on each defect category feature map, and determining at least one target location feature map based on each defect location feature map, wherein each target category feature map is a defect category feature map that meets a first resolution requirement among each defect category feature map, and each target location feature map is a defect location feature map that meets a second resolution requirement among each defect location feature map;

[0009] According to each target position feature map, a regression frame corresponding to each target position feature map is determined. For each regression frame, the area corresponding to the regression frame is the area where the defect corresponding to the target position feature map corresponding to the regression frame is located on the pipeline inner wall image;

[0010] Determining, based on each of the regression frames and each of the target category feature maps, classification information corresponding to each of the target category feature maps and a regression frame feature map corresponding to each of the regression frames, wherein the classification information is the category of the defect in the target category feature map, and for each of the regression frames, the regression frame feature map is a feature map of the region where the defect is located corresponding to the regression frame;

[0011] Determining category information corresponding to the pipeline inner wall image based on each piece of classification information, wherein the category information is a category of defects in the pipeline inner wall image;

[0012] According to each regression frame feature map, the regression frame score corresponding to each regression frame feature map is determined. The regression frame score represents the accuracy of the regression frame.

[0013] According to the regression frame scores corresponding to each regression frame and each classification information, the positioning information corresponding to the pipeline inner wall image is determined, and the positioning information is the positioning of the defect in the pipeline inner wall image.

[0014] The beneficial effect of a pipeline inner wall defect detection method provided by the present invention is: at least one target category feature map is determined by each defect category feature map, and at least one target position feature map is determined by each defect position, so that the defect classification and defect positioning corresponding to multiple scales of the defect on the pipeline inner wall image can be obtained, and then a regression frame is obtained according to at least one target category feature map, and classification information corresponding to each target category feature map is obtained according to the regression frame, so that the category of the defect on the pipeline inner wall can be obtained. In addition, a regression frame score is obtained according to the regression frame, and the accuracy of the regression frame can be determined, thereby improving the accuracy of defect positioning. Finally, the positioning of the defect on the pipeline inner wall can be obtained according to the regression frame and the score corresponding to the regression frame, which solves the problem that the existing technology cannot display the defect information in the pipeline and the positioning of the defect in the pipeline to front-line workers, which is not conducive to subsequent pipeline maintenance and repair.

[0015] On the basis of the above technical solution, the pipeline inner wall defect detection method of the present invention can be further improved as follows.

[0016] Furthermore, the above-mentioned determining, based on the pipeline inner wall image, a plurality of defect category feature maps and a plurality of defect position feature maps corresponding to the pipeline inner wall image includes:

[0017] The pipeline inner wall image is input into the feature extractor. Through each layer of the feature extractor, the defect category features of the pipeline inner wall image are extracted to obtain a defect category feature map, and the defect position features of the pipeline inner wall image are extracted to obtain a defect position feature map. Each layer in the feature extractor corresponds to a different resolution.

[0018] The beneficial effect of adopting the above further scheme is: by extracting the defect category features and defect position features of the pipeline inner wall image through different layers of the feature extractor, defect category feature maps and defect position feature maps of defects at different sizes (different resolutions) can be obtained, thereby improving the accuracy of defect classification and positioning.

[0019] Furthermore, the above-mentioned determining at least one target category feature map based on each defect category feature map, and determining at least one target location feature map based on each defect location feature map, includes:

[0020] S11, obtaining a defect category feature map and a defect position feature map corresponding to each of at least two feature extraction layers in a feature extractor, wherein each feature extraction layer in the feature extractor has a different resolution;

[0021] S12, starting from the first feature extraction layer of the at least two feature extraction layers to the last feature extraction layer, taking the first feature extraction layer as the current feature extraction layer, and repeatedly performing step S13 until the last feature extraction layer is taken as the current feature extraction layer, thereby obtaining a second category feature map and a second position feature map corresponding to each feature extraction layer;

[0022] S13, downsampling the defect category feature map and the defect position feature map corresponding to the current feature extraction layer to obtain a first category feature map and a first position feature map, obtaining a second category feature map based on the first category feature map and the defect category feature map corresponding to the first feature extraction layer, obtaining a second position feature map based on the first position feature map and the defect position feature map corresponding to the first feature extraction layer, and using the next layer of the current feature extraction layer as a new current feature extraction layer;

[0023] S14: Using the second category feature map corresponding to each feature extraction layer as at least one target category feature map, and using the second position feature map corresponding to each feature extraction layer as at least one target position feature map.

[0024] The beneficial effect of adopting the above further solution is: through S11-S14, each target category feature map and target position feature map flow between different feature extraction layers, thereby improving the accuracy of defect classification and positioning.

[0025] Furthermore, the above-mentioned step of determining the regression frame corresponding to each target position feature map according to each target position feature map includes:

[0026] According to each target position feature map, the regression frame corresponding to each target position feature map is determined through the regression branch, and the regression branch includes four first feature extraction modules and one regression module connected in sequence;

[0027] Among them, according to each target position feature map, the regression frame corresponding to each target position feature map is determined through the regression branch, including:

[0028] S21, for each target position feature map, starting from the first first feature extraction module to the last first feature extraction module, taking the first first feature extraction module as the first current module, and repeatedly performing step S22 until the last first feature extraction module is taken as the first current module, thereby obtaining a fifth position feature map corresponding to the last first feature extraction module;

[0029] S22, for each target position feature map, input the target position feature map into the first current module, extract the defect position feature of the target position feature map, obtain a fifth position feature map corresponding to the first current module, use the next first feature extraction module corresponding to the first current module as the new first current module, and use the fifth position feature map corresponding to the first current module as the target position feature map corresponding to the new first current module;

[0030] S23: For each target position feature map, input the fifth position feature map corresponding to the last first feature extraction module into the regression module to obtain a regression frame corresponding to the fifth position feature map.

[0031] The beneficial effect of adopting the above further scheme is: through multiple first feature extraction modules of the regression branch, the defect position features of the target position feature map are extracted, and finally the fifth position feature map is obtained, and the regression frame corresponding to the fifth position feature map can be obtained through the regression module, so as to find the defect area on the pipeline inner wall image.

[0032] Furthermore, the above-mentioned determination of the classification information corresponding to each target category feature map and the regression frame feature map corresponding to each regression frame based on each regression frame and each target category feature map includes:

[0033] According to each regression frame and each target category feature map, the classification information corresponding to each target category feature map and the regression frame feature map corresponding to each regression frame are determined through the classification branch, wherein the classification branch includes four second feature extraction modules connected in sequence, and the last second feature extraction module is respectively connected to the classification module and the regression frame feature map extraction module;

[0034] Each regression frame and each target category feature map, through the classification branch to determine the classification information corresponding to each target category feature map, and the regression frame feature map corresponding to each regression frame, including:

[0035] S31, for each target category feature map, starting from the first second feature extraction module to the last second feature extraction module, taking the first second feature extraction module as the second current module, and repeatedly performing step S32 until the last second feature extraction module is taken as the second current module, thereby obtaining a fifth category feature map corresponding to the last second feature extraction module;

[0036] S32: For each target category feature map, input the target category feature map into the second current module, extract the defect category features of the target category feature map, obtain a fifth category feature map corresponding to the second current module, use the next second feature extraction module corresponding to the second current module as a new second current module, and use the fifth category feature map corresponding to the second current module as a new target category feature map corresponding to the second current module;

[0037] S33, for each target category feature map, inputting the fifth category feature map corresponding to the last second feature extraction module into the classification module to obtain classification information corresponding to the target category feature map;

[0038] S34, for the regression box corresponding to each target category feature map, input the regression box into the regression box feature map extraction module, extract the features of the regression box, and obtain the regression box feature map.

[0039] The beneficial effect of adopting the above further scheme is: through multiple second feature extraction modules of the classification branch, the defect category features of the target category feature map are extracted to obtain the fifth category feature map, and then through the classification module, the classification information corresponding to each target category feature map can be obtained, and then the regression frame is input into the regression frame feature map extraction module to extract the regression frame features, which facilitates the subsequent judgment of the accuracy of the regression frame.

[0040] Furthermore, according to each regression frame feature map, the regression frame score corresponding to each regression frame feature map is determined, including:

[0041] Determine, according to each regression box feature map, a regression box score corresponding to each regression box feature map through a regression box quality module, wherein the regression box quality module includes a first branch and a second branch, and further, the first branch includes a first maximum pooling layer, and the second branch includes a grouped convolution and a second maximum pooling layer;

[0042] Among them, according to each regression frame feature map, the regression frame score corresponding to each regression frame feature map is determined by the regression frame quality module, including:

[0043] S41, for each regression frame feature map, input the regression frame feature map into the first maximum pooling layer for pooling operation to obtain a first target image;

[0044] S42, for each regression frame feature map, input the regression frame feature map into the volume group, extract features of the regression frame feature map, and obtain a second target image;

[0045] S43, for each regression box feature map, input the second target image into the second maximum pooling layer for pooling operation to obtain a third target image;

[0046] S44: For each regression box feature map, multiply the first target image and the third target image to obtain a regression box score.

[0047] The beneficial effect of adopting the above further scheme is: through the first branch and the second branch, the regression frame score corresponding to the regression frame is determined, and then a high-quality regression frame is obtained, and the low-quality regression frame is deleted, thereby improving the accuracy of defect classification and positioning.

[0048] In a second aspect, the present invention provides a pipeline inner wall defect detection system, comprising:

[0049] An image acquisition module, used for acquiring an image of the inner wall of the pipeline;

[0050] A feature map acquisition module is used to determine, based on the pipeline inner wall image, a plurality of defect category feature maps and a plurality of defect position feature maps corresponding to the pipeline inner wall image;

[0051] a target feature map acquisition module, configured to determine at least one target category feature map based on each defect category feature map, and to determine at least one target location feature map based on each defect location feature map, wherein each target category feature map is a defect category feature map that satisfies a first resolution requirement among each defect category feature map, and each target location feature map is a defect location feature map that satisfies a second resolution requirement among each defect location feature map;

[0052] The regression frame acquisition module is used to determine the regression frame corresponding to each target position feature map based on each target position feature map. For each regression frame, the area corresponding to the regression frame is the area where the defect corresponding to the target position feature map corresponding to the regression frame is located on the pipeline inner wall image;

[0053] A classification information acquisition module is used to determine, based on each regression frame and each target category feature map, the classification information corresponding to each target category feature map and the regression frame feature map corresponding to each regression frame. The classification information is the category of the defect in the target category feature map. For each regression frame, the regression frame feature map is the feature map of the defect area corresponding to the regression frame.

[0054] a category information acquisition module, configured to determine category information corresponding to the pipeline inner wall image based on each piece of classification information, wherein the category information is the category of defects in the pipeline inner wall image;

[0055] The regression box score acquisition module is used to determine the regression box score corresponding to each regression box feature map based on each regression box feature map. The regression box score represents the accuracy of the regression box;

[0056] The positioning information acquisition module is used to determine the positioning information corresponding to the pipeline inner wall image based on the regression frame scores corresponding to each regression frame and each classification information. The positioning information is the positioning of the defect in the pipeline inner wall image.

[0057] In a third aspect, the present invention further provides an electronic device comprising a memory, a processor, and a program stored in the memory and running on the processor, wherein when the processor executes the program, the steps of the above-mentioned method for detecting defects in the inner wall of a pipeline are implemented.

[0058] In a fourth aspect, the present invention further provides a computer-readable storage medium, in which instructions are stored. When the instructions are executed on a terminal device, the terminal device executes the steps of a method for detecting defects in the inner wall of a pipeline. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the present invention is further described below with reference to the accompanying drawings and embodiments.

[0060] Figure 1 Schematic diagram of the structure of the internal detector according to an embodiment of the present invention;

[0061] Figure 2 This is a flow chart of a pipeline inner wall defect detection method according to an embodiment of the present invention;

[0062] Figure 3 This is a framework diagram of a pipeline inner wall defect detection method according to an embodiment of the present invention;

[0063] Figure 4 This is the framework diagram of the Head module;

[0064] Figure 5 This is the framework diagram of the regression box quality module;

[0065] Figure 6 It is the framework diagram of the space joint pyramid;

[0066] Figure 7 The figure is a schematic structural diagram of a pipeline inner wall defect detection system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0067] The following examples are provided to further explain and supplement the present invention and do not constitute any limitation to the present invention.

[0068] A pipeline inner wall defect detection method and system according to an embodiment of the present invention will be described below with reference to the accompanying drawings.

[0069] A method for detecting defects in the inner wall of a pipeline in an embodiment of the present invention is applied to a terminal device. In this application scheme, the terminal device is used as the execution subject to illustrate the application scheme. The terminal device can be a computer, server, etc., which is used to execute the steps of a method for detecting defects in the inner wall of a pipeline.

[0070] like Figure 1 As shown, the inner wall of the pipeline is inspected by an internal detector, wherein the internal detector includes 9 circumferential industrial HD cameras to obtain images of the inner wall of the pipeline, and two adjacent industrial HD cameras are separated by 40 degrees. In addition, the internal detector is also equipped with a thermal imager and a forward light source. The thermal imager is used to obtain thermal imaging images, thereby displaying thermal imaging images to front-line workers, and the forward light source is used for fill light, so that the industrial HD camera can obtain clear images of the inner wall of the pipeline.

[0071] like Figure 2 As shown, the present invention provides a method for detecting defects on the inner wall of a pipeline, comprising the following steps:

[0072] S1, obtain the image of the inner wall of the pipeline;

[0073] S2, determining, based on the pipeline inner wall image, a plurality of defect category feature maps and a plurality of defect position feature maps corresponding to the pipeline inner wall image;

[0074] S3, determining at least one target category feature map based on each defect category feature map, and determining at least one target location feature map based on each defect location feature map, wherein each target category feature map is a defect category feature map that meets the first resolution requirement among the defect category feature maps, and each target location feature map is a defect location feature map that meets the second resolution requirement among the defect location feature maps;

[0075] S4, determining a regression frame corresponding to each target position feature map based on each target position feature map, wherein for each regression frame, the area corresponding to the regression frame is the area where the defect corresponding to the target position feature map corresponding to the regression frame is located on the pipeline inner wall image;

[0076] S5, determining, based on each of the regression frames and each of the target category feature maps, classification information corresponding to each of the target category feature maps, and a regression frame feature map corresponding to each of the regression frames, wherein the classification information is the category of the defect in the target category feature map, and for each of the regression frames, the regression frame feature map is a feature map of the region where the defect is located corresponding to the regression frame;

[0077] S6, determining category information corresponding to the pipeline inner wall image based on each piece of classification information, where the category information is the category of defects in the pipeline inner wall image;

[0078] S7, according to each regression frame feature map, determining the regression frame score corresponding to each regression frame feature map, the regression frame score represents the accuracy of the regression frame;

[0079] S8. Determine positioning information corresponding to the pipeline inner wall image based on the regression frame scores corresponding to the regression frames and the classification information. The positioning information is the positioning of the defect in the pipeline inner wall image.

[0080] Optionally, the determining, based on the pipeline inner wall image, a plurality of defect category feature maps and a plurality of defect position feature maps corresponding to the pipeline inner wall image includes:

[0081] The pipeline inner wall image is input into the feature extractor. Through each layer of the feature extractor, the defect category features of the pipeline inner wall image are extracted to obtain a defect category feature map, and the defect position features of the pipeline inner wall image are extracted to obtain a defect position feature map. Each layer in the feature extractor corresponds to a different resolution.

[0082] In this embodiment, Figure 3 As shown, the image of the inner wall of the pipeline is input into the feature extractor, and the defect category feature map or defect position feature map corresponding to each layer of the feature extractor (also called the feature extraction layer) is obtained through the feature extractor. Since the resolution corresponding to each layer in the feature extractor is different, the image resolution obtained in each layer is different. However, if the feature map of each resolution is processed to obtain the defect category and position, it will not only increase the amount of calculation, but also cause the image to be unclear if the resolution is too large or too small, which is not conducive to the subsequent classification and positioning of defects. Therefore, in this embodiment, the defect category feature map or defect position feature map (B3-B5, P6-P7) corresponding to the third to seventh layers of the feature extractor can be selected for subsequent processing. In addition, each layer of the feature extractor can obtain a defect category feature map or a defect position feature map. For example, B5 can represent both a defect category feature map and a defect position feature map.

[0083] Optionally, the determining of at least one target category feature map based on each defect category feature map, and the determining of at least one target location feature map based on each defect location feature map, include:

[0084] S11, obtaining a defect category feature map and a defect position feature map corresponding to each of at least two feature extraction layers in a feature extractor, wherein each feature extraction layer in the feature extractor has a different resolution;

[0085] S12, starting from the first feature extraction layer of the at least two feature extraction layers to the last feature extraction layer, taking the first feature extraction layer as the current feature extraction layer, and repeatedly performing step S13 until the last feature extraction layer is taken as the current feature extraction layer, thereby obtaining a second category feature map and a second position feature map corresponding to each feature extraction layer;

[0086] S13, downsampling the defect category feature map and the defect position feature map corresponding to the current feature extraction layer to obtain a first category feature map and a first position feature map, obtaining a second category feature map based on the first category feature map and the defect category feature map corresponding to the first feature extraction layer, obtaining a second position feature map based on the first position feature map and the defect position feature map corresponding to the first feature extraction layer, and using the next layer of the current feature extraction layer as a new current feature extraction layer;

[0087] S14: Using the second category feature map corresponding to each feature extraction layer as at least one target category feature map, and using the second position feature map corresponding to each feature extraction layer as at least one target position feature map.

[0088] Optional, such as Figure 3 As shown, in this embodiment, the third to fifth layers of the feature extractor are selected as feature extraction layers for obtaining target category feature maps and target position feature maps. Optionally, the fifth layer of the feature extractor is selected as the first feature extraction layer, and the third layer of the feature extractor is selected as the last feature extraction layer. Execute S12-S13.

[0089] Optional, such as Figure 3 As shown, in S13, when the third layer of the feature extractor is used as the current feature extraction layer, the defect category feature map and the defect position feature map (B5 or P5) corresponding to the current feature extraction layer are downsampled to obtain a first category feature map and a first position feature map. At this time, since the first category feature map and the defect category feature map corresponding to the fourth layer of the feature extractor have the same resolution, and the first position feature map and the defect position feature map corresponding to the fourth layer of the feature extractor have the same resolution, the first category feature map and the defect category feature map corresponding to the fourth layer of the feature extractor are added by pixel values ​​to obtain the second category feature map (P4) corresponding to the current feature layer, and the first position feature map and the defect position feature map corresponding to the fourth layer of the feature extractor are added by pixel values ​​to obtain the second position feature map (P4) corresponding to the current feature layer. By analogy, the above method is repeated to obtain the second category feature map and the second position feature map corresponding to each current feature extraction layer.

[0090] In addition, if Figure 3As shown, it should be noted that at least one target category feature map includes not only the second category feature map (P3 and P4) corresponding to each feature extraction layer, but also the defect category feature maps (B5, P6 and P7) corresponding to the fifth to seventh layers of the feature extractor. Similarly, at least one target position feature map includes not only the second position feature map (P3 and P4) corresponding to each feature extraction layer, but also the defect position feature maps (B5, P6 and P7) corresponding to the fifth to seventh layers of the feature extractor.

[0091] Optional, such as Figure 3 As shown in , after obtaining the feature maps of each target category and each target position, a category space pyramid and a regression space pyramid are constructed, and the category of the defect is obtained by using the category space pyramid, and the location of the defect is obtained by using the regression space pyramid. However, before the category space pyramid and the regression space pyramid, an additional Head module is added to obtain the regression box and the regression box score corresponding to the regression box. Optionally, as shown in Figure 4 As shown, the Head module includes a classification branch ( Figure 4 The first row in ) and the regression branch ( Figure 4 The second row in the figure), where the classification branch (Classification) includes four second feature extraction modules (3×3×256) connected in sequence, and the last second feature extraction module is connected to the classification module (3×3×n) and the regression box feature map extraction module ( Figure 4 The center-ness corresponding to the dotted line is connected, and the regression branch (Regression) includes four first feature extraction modules (3×3×256) and one regression module (3×3×4) connected in sequence.

[0092] Optionally, the above-mentioned determining the regression frame corresponding to each target position feature map according to each target position feature map includes:

[0093] According to each target position feature map, the regression frame corresponding to each target position feature map is determined through the regression branch, and the regression branch includes four first feature extraction modules and one regression module connected in sequence;

[0094] Among them, according to each target position feature map, the regression frame corresponding to each target position feature map is determined through the regression branch, including:

[0095] S21, for each target position feature map, starting from the first first feature extraction module to the last first feature extraction module, taking the first first feature extraction module as the first current module, and repeatedly performing step S22 until the last first feature extraction module is taken as the first current module, thereby obtaining a fifth position feature map corresponding to the last first feature extraction module;

[0096] S22, for each target position feature map, input the target position feature map into the first current module, extract the defect position feature of the target position feature map, obtain a fifth position feature map corresponding to the first current module, use the next first feature extraction module corresponding to the first current module as the new first current module, and use the fifth position feature map corresponding to the first current module as the target position feature map corresponding to the new first current module;

[0097] S23: For each target position feature map, input the fifth position feature map corresponding to the last first feature extraction module into the regression module to obtain a regression frame corresponding to the fifth position feature map.

[0098] Optionally, the above-mentioned determination of the classification information corresponding to each target category feature map and the regression frame feature map corresponding to each regression frame based on each regression frame and each target category feature map includes:

[0099] According to each regression frame and each target category feature map, the classification information corresponding to each target category feature map and the regression frame feature map corresponding to each regression frame are determined through the classification branch, wherein the classification branch includes four second feature extraction modules connected in sequence, and the last second feature extraction module is respectively connected to the classification module and the regression frame feature map extraction module;

[0100] According to each regression frame and each target category feature map, the classification information corresponding to each target category feature map is determined, including:

[0101] S31, for each target category feature map, starting from the first second feature extraction module to the last second feature extraction module, taking the first second feature extraction module as the second current module, and repeatedly performing step S32 until the last second feature extraction module is taken as the second current module, thereby obtaining a fifth category feature map corresponding to the last second feature extraction module;

[0102] S32: For each target category feature map, input the target category feature map into the second current module, extract the defect category features of the target category feature map, obtain a fifth category feature map corresponding to the second current module, use the next second feature extraction module corresponding to the second current module as a new second current module, and use the fifth category feature map corresponding to the second current module as a new target category feature map corresponding to the second current module;

[0103] S33, for each target category feature map, inputting the fifth category feature map corresponding to the last second feature extraction module into the classification module to obtain classification information corresponding to the target category feature map;

[0104] S34, for the regression box corresponding to each target category feature map, input the regression box into the regression box feature map extraction module, extract the features of the regression box, and obtain the regression box feature map.

[0105] Optionally, after obtaining the classification information corresponding to each target category feature map through S33, the classification information corresponding to each target category feature map can be input into the corresponding level of the category space pyramid, and the category information corresponding to the final pipeline inner wall image can be obtained through the category space pyramid.

[0106] Optionally, the purpose of adding a regression frame feature map extraction module in the classification branch is to further extract the features of the regression frame, obtain the regression frame feature map, and then map it to the regression space for judging the quality of the regression frame (regression frame score), that is, judging whether the center of the regression frame is near the defect center, so as to discard the regression frame whose center deviates greatly from the defect center, thereby improving the accuracy of defect positioning. Based on this, according to each regression frame feature map, the regression frame score corresponding to each regression frame feature map is determined, including:

[0107] Determine, according to each regression box feature map, a regression box score corresponding to each regression box feature map through a regression box quality module, wherein the regression box quality module includes a first branch and a second branch, and further, the first branch includes a first maximum pooling layer, and the second branch includes a grouped convolution and a second maximum pooling layer;

[0108] Among them, according to each regression frame feature map, the regression frame score corresponding to each regression frame feature map is determined by the regression frame quality module, including:

[0109] S41, for each regression frame feature map, input the regression frame feature map into the first maximum pooling layer for pooling operation to obtain a first target image;

[0110] S42, for each regression frame feature map, input the regression frame feature map into the volume group, extract features of the regression frame feature map, and obtain a second target image;

[0111] S43, for each regression box feature map, input the second target image into the second maximum pooling layer for pooling operation to obtain a third target image;

[0112] S44: For each regression box feature map, multiply the first target image and the third target image to obtain a regression box score.

[0113] Optional, such as Figure 5 As shown, the first branch in the regression box quality module is Figure 5 The first maximum pooling layer (Objectness map) in the second branch is the group convolution and the second maximum pooling layer (CenTerness map), which multiplies the first target image and the second target image ( Figure 5 middle ), you can get the regression box score.

[0114] Optional, such as Figure 6 As shown, the output result of each layer of the regression box quality module (bbox quality attention) needs to fuse the classification information of the corresponding layer of the category space pyramid. For example, the input of the first layer R'7 of the regression space pyramid is: the classification information corresponding to the target category feature map corresponding to the category space pyramid C'7 and the classification information corresponding to the first and second layers of the regression box quality module ( Figure 6 The fusion results of the regression frame quality module from top to bottom are the first, second, third, fourth and fifth layers. The fusion results of the first and second layers of the regression frame quality module are the regression frame scores corresponding to the first layer and the regression frame scores corresponding to the second layer of the regression frame quality module, which are added according to the preset weights. Finally, after obtaining the input corresponding to each layer of the regression space pyramid, the positioning information corresponding to the pipeline inner wall image can be obtained through the regression space pyramid.

[0115] like Figure 7 As shown, an embodiment of the present invention further provides a pipeline inner wall defect detection system, comprising:

[0116] An image acquisition module 201 is used to acquire an image of the inner wall of a pipeline;

[0117] A feature map acquisition module 202 is configured to determine, based on the pipeline inner wall image, a plurality of defect category feature maps and a plurality of defect position feature maps corresponding to the pipeline inner wall image;

[0118] The target feature map acquisition module 203 is configured to determine at least one target category feature map based on each defect category feature map, and to determine at least one target location feature map based on each defect location feature map, wherein each target category feature map is a defect category feature map that satisfies a first resolution requirement among each defect category feature map, and each target location feature map is a defect location feature map that satisfies a second resolution requirement among each defect location feature map;

[0119] The regression frame acquisition module 204 is used to determine the regression frame corresponding to each target position feature map based on each target position feature map. For each regression frame, the area corresponding to the regression frame is the area where the defect corresponding to the target position feature map corresponding to the regression frame is located on the pipeline inner wall image;

[0120] The classification information acquisition module 205 is used to determine the classification information corresponding to each target category feature map and the regression frame feature map corresponding to each regression frame based on each regression frame and each target category feature map. The classification information is the category of the defect in the target category feature map. For each regression frame, the regression frame feature map is the feature map of the defect area corresponding to the regression frame.

[0121] A category information acquisition module 206 is configured to determine category information corresponding to the pipeline inner wall image based on each piece of classification information, where the category information is the category of defects in the pipeline inner wall image;

[0122] A regression frame score acquisition module 207 is used to determine the regression frame score corresponding to each regression frame feature map based on each regression frame feature map, where the regression frame score represents the accuracy of the regression frame;

[0123] The positioning information acquisition module 208 is used to determine the positioning information corresponding to the pipeline inner wall image according to the regression frame scores corresponding to each regression frame and each classification information. The positioning information is the location of the defect in the pipeline inner wall image.

[0124] Optionally, the image acquisition module 201 is specifically configured to:

[0125] The pipeline inner wall image is input into the feature extractor. Through each layer of the feature extractor, the defect category features of the pipeline inner wall image are extracted to obtain a defect category feature map, and the defect position features of the pipeline inner wall image are extracted to obtain a defect position feature map. Each layer in the feature extractor corresponds to a different resolution.

[0126] Optionally, the feature map acquisition module 202 determines at least one target category feature map and at least one target position feature map through a first unit, wherein the first unit is specifically configured to:

[0127] S11, obtaining a defect category feature map and a defect position feature map corresponding to each of at least two feature extraction layers in a feature extractor, wherein each feature extraction layer in the feature extractor has a different resolution;

[0128] S12, starting from the first feature extraction layer of the at least two feature extraction layers to the last feature extraction layer, taking the first feature extraction layer as the current feature extraction layer, and repeatedly performing step S13 until the last feature extraction layer is taken as the current feature extraction layer, thereby obtaining a second category feature map and a second position feature map corresponding to each feature extraction layer;

[0129] S13, downsampling the defect category feature map and the defect position feature map corresponding to the current feature extraction layer to obtain a first category feature map and a first position feature map, obtaining a second category feature map based on the first category feature map and the defect category feature map corresponding to the first feature extraction layer, obtaining a second position feature map based on the first position feature map and the defect position feature map corresponding to the first feature extraction layer, and using the next layer of the current feature extraction layer as a new current feature extraction layer;

[0130] S14: Using the second category feature map corresponding to each feature extraction layer as at least one target category feature map, and using the second position feature map corresponding to each feature extraction layer as at least one target position feature map.

[0131] Optionally, the regression frame acquisition module 204 determines the regression frame corresponding to each target position feature map through the second unit, and the second unit is specifically used to:

[0132] S21, for each target position feature map, starting from the first first feature extraction module to the last first feature extraction module, taking the first first feature extraction module as the first current module, and repeatedly performing step S22 until the last first feature extraction module is taken as the first current module, thereby obtaining a fifth position feature map corresponding to the last first feature extraction module;

[0133] S22, for each target position feature map, input the target position feature map into the first current module, extract the defect position feature of the target position feature map, obtain a fifth position feature map corresponding to the first current module, use the next first feature extraction module corresponding to the first current module as the new first current module, and use the fifth position feature map corresponding to the first current module as the target position feature map corresponding to the new first current module;

[0134] S23: For each target position feature map, input the fifth position feature map corresponding to the last first feature extraction module into the regression module to obtain a regression frame corresponding to the fifth position feature map.

[0135] Optionally, the classification information acquisition module 205 determines the classification information corresponding to each target category feature map and the regression frame feature map corresponding to each regression frame through the third unit, wherein the third unit is specifically used to:

[0136] S31, for each target category feature map, starting from the first second feature extraction module to the last second feature extraction module, taking the first second feature extraction module as the second current module, and repeatedly performing step S32 until the last second feature extraction module is taken as the second current module, thereby obtaining a fifth category feature map corresponding to the last second feature extraction module;

[0137] S32: For each target category feature map, input the target category feature map into the second current module, extract the defect category features of the target category feature map, obtain a fifth category feature map corresponding to the second current module, use the next second feature extraction module corresponding to the second current module as a new second current module, and use the fifth category feature map corresponding to the second current module as a new target category feature map corresponding to the second current module;

[0138] S33, for each target category feature map, inputting the fifth category feature map corresponding to the last second feature extraction module into the classification module to obtain classification information corresponding to the target category feature map;

[0139] S34, for the regression box corresponding to each target category feature map, input the regression box into the regression box feature map extraction module, extract the features of the regression box, and obtain the regression box feature map.

[0140] Optionally, the regression box score acquisition module 207 determines the regression box score corresponding to each regression box feature map through the fourth unit, wherein the fourth unit is specifically used to:

[0141] S41, for each regression frame feature map, input the regression frame feature map into the first maximum pooling layer for pooling operation to obtain a first target image;

[0142] S42, for each regression frame feature map, input the regression frame feature map into the volume group, extract features of the regression frame feature map, and obtain a second target image;

[0143] S43, for each regression box feature map, input the second target image into the second maximum pooling layer for pooling operation to obtain a third target image;

[0144] S44: For each regression box feature map, multiply the first target image and the third target image to obtain a regression box score.

[0145] An electronic device according to an embodiment of the present invention includes a memory, a processor, and a program stored in the memory and running on the processor. When the processor executes the program, some or all steps of the above-mentioned method for detecting inner wall defects of a pipeline are implemented.

[0146] Among them, the electronic device can be a computer, and correspondingly, its program is computer software. The above-mentioned parameters and steps in an electronic device of the present invention can refer to the parameters and steps in the embodiment of a pipeline inner wall defect detection method above, and will not be repeated here.

[0147] Those skilled in the art will appreciate that the present invention may be implemented as a system, method, or computer program product. Therefore, the present disclosure may be specifically implemented in the following forms, namely: in the form of complete hardware, complete software (including firmware, resident software, microcode, etc.), or a combination of hardware and software, generally referred to herein as a "circuit," "module," or "system." Furthermore, in some embodiments, the present invention may also be implemented in the form of a computer program product in one or more computer-readable media, the computer-readable media containing computer-readable program code. Computer-readable storage media may be, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or components, or any combination thereof.

[0148] In the description of this specification, the reference terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" mean that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0149] Although the embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are illustrative and are not to be construed as limitations on the present invention. A person skilled in the art may change, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. A method for detecting defects on the inner wall of a pipeline, characterized in that: The following steps are involved: Acquire images of the inner wall of the pipeline; Determining, based on the pipeline inner wall image, a plurality of defect category feature maps and a plurality of defect position feature maps corresponding to the pipeline inner wall image; Determining at least one target category feature map based on each of the defect category feature maps, and determining at least one target location feature map based on each of the defect location feature maps, wherein each of the target category feature maps is a defect category feature map that meets a first resolution requirement among the defect category feature maps, and each of the target location feature maps is a defect location feature map that meets a second resolution requirement among the defect location feature maps; Determine, based on each of the target position feature maps, a regression frame corresponding to each of the target position feature maps, wherein for each of the regression frames, an area corresponding to the regression frame is an area where a defect corresponding to the target position feature map corresponding to the regression frame is located on the pipeline inner wall image; Determining, based on each of the regression frames and each of the target category feature maps, classification information corresponding to each of the target category feature maps and a regression frame feature map corresponding to each of the regression frames, wherein the classification information is the category of the defect in the target category feature map, and for each of the regression frames, the regression frame feature map is a feature map of the region where the defect is located corresponding to the regression frame; Determining category information corresponding to the pipeline inner wall image based on each piece of classification information, wherein the category information is a category of defects in the pipeline inner wall image; Determine, according to each of the regression frame feature maps, a regression frame score corresponding to each of the regression frame feature maps, wherein the regression frame score represents the accuracy of the regression frame; According to the regression frame scores corresponding to the respective regression frames and the respective classification information, positioning information corresponding to the pipeline inner wall image is determined, where the positioning information is the positioning of the defect in the pipeline inner wall image.

2. The method according to claim 1, characterized in that The determining, based on the pipeline inner wall image, a plurality of defect category feature maps and a plurality of defect position feature maps corresponding to the pipeline inner wall image includes: The pipeline inner wall image is input into a feature extractor. Through each layer of the feature extractor, defect category features of the pipeline inner wall image are extracted to obtain the defect category feature map, and defect position features of the pipeline inner wall image are extracted to obtain the defect position feature map. Each layer in the feature extractor corresponds to a different resolution.

3. The method according to claim 2, characterized in that Determining at least one target category feature map based on each of the defect category feature maps, and determining at least one target location feature map based on each of the defect location feature maps, includes: S11, obtaining the defect category feature map and the defect position feature map corresponding to each of the at least two feature extraction layers in the feature extractor, wherein the feature extraction layers in the feature extractor have different resolutions; S12, starting from the first feature extraction layer of the at least two feature extraction layers to the last feature extraction layer, taking the first feature extraction layer as the current feature extraction layer, and repeatedly performing step S13 until the last feature extraction layer is used as the current feature extraction layer, to obtain a second category feature map and a second position feature map corresponding to each feature extraction layer; S13, downsampling the defect category feature map and the defect position feature map corresponding to the current feature extraction layer to obtain a first category feature map and a first position feature map, obtaining a second category feature map based on the first category feature map and the defect category feature map corresponding to the first feature extraction layer, obtaining a second position feature map based on the first position feature map and the defect position feature map corresponding to the first feature extraction layer, and using the next layer of the current feature extraction layer as a new current feature extraction layer; S14: Using the second category feature map corresponding to each of the feature extraction layers as the at least one target category feature map, and using the second position feature map corresponding to each of the feature extraction layers as the at least one target position feature map.

4. The method according to any one of claims 1 to 3, characterized in that The step of determining, based on each of the target position feature maps, a regression frame corresponding to each of the target position feature maps, includes: According to each of the target position feature maps, determining a regression frame corresponding to each target position feature map through a regression branch, wherein the regression branch includes four first feature extraction modules and one regression module connected in sequence; Wherein, determining the regression frame corresponding to each target position feature map through the regression branch according to each target position feature map includes: S21, for each target position feature map, starting from the first first feature extraction module to the last first feature extraction module, taking the first first feature extraction module as the first current module, and repeatedly performing step S22 until the last first feature extraction module is taken as the first current module, thereby obtaining a fifth position feature map corresponding to the last first feature extraction module; S22, for each target position feature map, input the target position feature map into the first current module, extract the defect position features of the target position feature map, obtain a fifth position feature map corresponding to the first current module, use the next first feature extraction module corresponding to the first current module as a new first current module, and use the fifth position feature map corresponding to the first current module as the target position feature map corresponding to the new first current module; S23: For each of the target position feature maps, input the fifth position feature map corresponding to the last one of the first feature extraction modules into the regression module to obtain a regression frame corresponding to the fifth position feature map.

5. The method according to any one of claims 1 to 3, characterized in that The determining, according to each of the regression frames and each of the target category feature maps, classification information corresponding to each of the target category feature maps and a regression frame feature map corresponding to each of the regression frames includes: According to each of the regression frames and each of the target category feature maps, determining, through a classification branch, classification information corresponding to each of the target category feature maps and a regression frame feature map corresponding to each of the regression frames, wherein the classification branch includes four second feature extraction modules connected in sequence, and the last of the second feature extraction modules is respectively connected to a classification module and a regression frame feature map extraction module; The respective regression frames and the respective target category feature maps, determining classification information corresponding to each target category feature map and the regression frame feature map corresponding to each regression frame through a classification branch, including: S31, for each target category feature map, starting from the first second feature extraction module to the last second feature extraction module, taking the first second feature extraction module as the second current module, and repeatedly performing step S32 until the last second feature extraction module is taken as the second current module, thereby obtaining a fifth category feature map corresponding to the last second feature extraction module; S32: For each target category feature map, input the target category feature map into the second current module, extract the defect category features of the target category feature map, obtain a fifth category feature map corresponding to the second current module, use the next second feature extraction module corresponding to the second current module as a new second current module, and use the fifth category feature map corresponding to the second current module as the target category feature map corresponding to the new second current module; S33, for each target category feature map, inputting the fifth category feature map corresponding to the last one of the second feature extraction modules into the classification module to obtain classification information corresponding to the target category feature map; S34: For each regression frame corresponding to the target category feature map, input the regression frame into the regression frame feature map extraction module, extract the features of the regression frame, and obtain the regression frame feature map.

6. The method according to any one of claims 1 to 3, characterized in that Determining, according to each of the regression frame feature maps, a regression frame score corresponding to each of the regression frame feature maps, includes: Determine, according to each of the regression box feature maps, a regression box score corresponding to each of the regression box feature maps through a regression box quality module, wherein the regression box quality module includes a first branch and a second branch, and further, the first branch includes a first maximum pooling layer, and the second branch includes a grouped convolution and a second maximum pooling layer; Wherein, determining the regression box score corresponding to each regression box feature map through the regression box quality module according to each regression box feature map includes: S41, for each of the regression frame feature maps, input the regression frame feature map into a first maximum pooling layer for pooling operation to obtain a first target image; S42, for each of the regression frame feature maps, inputting the regression frame feature map into a volume group, extracting features of the regression frame feature map, and obtaining a second target image; S43: For each of the regression box feature maps, input the second target image into a second maximum pooling layer for pooling operation to obtain a third target image; S44: For each of the regression frame feature maps, multiply the first target image and the third target image to obtain a regression frame score.

7. A pipeline inner wall defect detection system, characterized in that: include: An image acquisition module, used for acquiring an image of the inner wall of the pipeline; a feature map acquisition module, configured to determine, based on the pipeline inner wall image, a plurality of defect category feature maps and a plurality of defect position feature maps corresponding to the pipeline inner wall image; a target feature map acquisition module, configured to determine at least one target category feature map based on each of the defect category feature maps, and to determine at least one target location feature map based on each of the defect location feature maps, wherein each of the target category feature maps is a defect category feature map that satisfies a first resolution requirement among the defect category feature maps, and each of the target location feature maps is a defect location feature map that satisfies a second resolution requirement among the defect location feature maps; A regression frame acquisition module is used to determine, based on each of the target position feature maps, a regression frame corresponding to each of the target position feature maps, wherein for each of the regression frames, the area corresponding to the regression frame is the area where the defect corresponding to the target position feature map corresponding to the regression frame is located on the pipeline inner wall image; a classification information acquisition module, configured to determine, based on each regression frame and each target category feature map, the classification information corresponding to each target category feature map and the regression frame feature map corresponding to each regression frame, wherein the classification information is the category of the defect in the target category feature map, and for each regression frame, the regression frame feature map is a feature map of the region where the defect is located corresponding to the regression frame; a category information acquisition module, configured to determine category information corresponding to the pipeline inner wall image based on each piece of classification information, wherein the category information is a category of defects in the pipeline inner wall image; A regression frame score acquisition module is used to determine the regression frame score corresponding to each regression frame feature map according to each regression frame feature map, wherein the regression frame score represents the accuracy of the regression frame; A positioning information acquisition module is used to determine the positioning information corresponding to the pipeline inner wall image based on the regression frame scores corresponding to the regression frames and the classification information, where the positioning information is the positioning of the defect in the pipeline inner wall image.

8. An electronic device comprising a memory, a processor, and a program stored in the memory and running on the processor, characterized in that: When the processor executes the program, the steps of the pipeline inner wall defect detection method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores instructions, and when the instructions are executed on a terminal device, the terminal device executes the steps of a pipeline inner wall defect detection method according to any one of claims 1 to 6.

Images