A pipeline quality inspection method and device based on panoramic vision

Abstract

The present application relates to a kind of pipeline quality inspection method and pipeline quality inspection device based on panoramic vision, it obtains the local picture of multiple groups of target pipeline wall first, then splices all the local picture in each group, obtains multiple local panoramic pictures, then splices multiple local panoramic pictures, obtains pipeline panoramic picture, finally, according to the pipeline panoramic picture, based on preset target detection model detects the pipeline panoramic picture, identifies and locates target defect.Compared with prior art, the present application obtains the pipeline panoramic picture of pipeline by the way of splicing local picture of multiple angles, then utilizes preset target detection model to complete detection, and the detection based on vision only needs to use camera, is cheap and has wide applicability, most work can be completed by computer software, except camera, there is no laser probe and other requirements for equipment accuracy, is less affected by environment, and detection result is stable, has good practicability.

Description

Technical Field

[0001] This invention relates to the field of industrial quality inspection technology, and in particular to a pipeline quality inspection method and device based on panoramic vision. Background Technology

[0002] In modern industrial safety operations and building construction, the application of pipelines in large-scale buildings and other fields is gradually expanding, and the quality of the pipeline surface has a crucial impact on the pipeline's service life. Defects that occur during pipeline transportation can directly affect the pipeline's use, leading to leaks, fires, or explosions of the transported medium, resulting in significant losses. Therefore, defect detection and treatment of pipelines are essential tasks.

[0003] Traditional testing often requires expensive equipment such as laser probes and ultrasonic probes. These devices are easily affected by complex environments in the workplace, which can affect the test results.

[0004] Therefore, there is an urgent need for a pipeline quality inspection method that is less affected by the environment and provides stable test results. Summary of the Invention

[0005] In view of this, it is necessary to provide a pipeline quality inspection method and pipeline quality inspection device based on panoramic vision to solve the problem that existing technologies are easily affected by complex environments in the workplace, thus affecting the inspection results.

[0006] To achieve the above-mentioned technical objectives, the present invention adopts the following technical solution:

[0007] In a first aspect, the present invention provides a pipeline quality inspection method based on panoramic vision, comprising:

[0008] Multiple sets of partial images of the target pipe wall are acquired. The multiple partial images in the same set correspond to the same axial position on the target pipe, and the shooting direction of the multiple partial images in the same set is arranged around the target pipe based on the corresponding axial position. The axial positions corresponding to different sets of partial images are different.

[0009] By stitching together all the local images in each group, multiple local panoramic images are obtained;

[0010] By stitching together multiple partial panoramic images, a panoramic image of the pipeline is obtained;

[0011] Based on the pipeline panoramic image, the pipeline panoramic image is detected using a preset target detection model to identify and locate target defects.

[0012] Furthermore, by stitching together all the partial images in each group, multiple partial panoramic images are obtained, including:

[0013] Each of the local images in the same group is preprocessed to obtain multiple optimized images;

[0014] Based on the actual positional relationships of the optimization graphs, the multiple optimization graphs in the same group are sorted.

[0015] In the same group, the i-th stitched image and the j-th optimized image are stitched together to obtain the (i+1)-th stitched image;

[0016] Wherein, i is the number of stitching, j is the sorting number of the multiple optimized images, i and j are both positive integers, and j = i + 2. The first stitched image is formed by stitching together the first optimized image and the second optimized image, and the final stitched image is the local panoramic image.

[0017] Furthermore, the preprocessing of each local image in the same group yields multiple optimized images, including:

[0018] Denoising the local image;

[0019] The pipe image is extracted from the denoised local image to obtain a local pipe map;

[0020] Perform a cylindrical back projection transformation on the local piping diagram to obtain the piping unfolded diagram;

[0021] The pipeline unfolded diagram is repaired and optimized to obtain the optimized diagram.

[0022] Furthermore, the step of sorting multiple optimization maps in the same group based on their actual positional relationships includes:

[0023] Based on the axial position corresponding to each group of optimization diagrams, the multiple groups of optimization diagrams are sorted among themselves.

[0024] Based on the surrounding order of the shooting direction corresponding to each of the optimized images in the same group, the optimized images in the kth group are sorted in ascending order;

[0025] Based on the surrounding order of the shooting direction corresponding to each of the optimized images in the same group, the optimized images in the (k+1)th group are sorted in reverse order;

[0026] Wherein, k refers to the number of times each group of the optimized graph is passed through.

[0027] Furthermore, the step of stitching the i-th stitched image and the j-th optimized image in the same group to obtain the (i+1)-th stitched image includes:

[0028] Extract feature descriptors from the i-th stitched image and the j-th optimized image;

[0029] Based on the feature descriptor, the perspective transformation matrix is ​​obtained;

[0030] According to the perspective transformation matrix, the i-th stitched image and / or the j-th optimized image are subjected to perspective transformation to obtain two images to be merged.

[0031] The optimal suture line is obtained based on the overlapping area of ​​the two images to be fused.

[0032] Based on the optimal suture line, the two images to be fused are fused together to obtain the (i+1)th spliced ​​image.

[0033] Furthermore, the step of detecting the pipeline panoramic image based on a preset target detection model, and identifying and locating target defects, includes:

[0034] Obtain a sample dataset characterizing pipeline defects;

[0035] A neural network model is established, and the neural network model is trained based on the sample dataset to obtain the preset target detection model;

[0036] The pipeline panoramic image is detected based on the preset target detection model to identify and locate the target defect.

[0037] Secondly, the present invention also provides a pipeline quality inspection device based on panoramic vision, applied to the pipeline quality inspection method based on panoramic vision described in any of the above claims, wherein the pipeline quality inspection device based on panoramic vision includes:

[0038] The image acquisition module is used to acquire multiple sets of partial images of the target pipe wall. Multiple partial images in the same set correspond to the same axial position on the target pipe, and the shooting direction of multiple partial images in the same set is arranged around the target pipe based on the corresponding axial position. The axial positions corresponding to different sets of partial images are different.

[0039] The primary stitching module is used to stitch together all the local images in each group to obtain multiple local panoramic images;

[0040] The secondary stitching module is used to stitch together multiple partial panoramic images to obtain a panoramic image of the pipeline.

[0041] The defect detection module is used to detect the pipeline panoramic image based on a preset target detection model, and identify and locate target defects.

[0042] Furthermore, the image acquisition module includes:

[0043] A pipe climbing assembly includes a clamping part, a first driving part, and a first driving wheel. The clamping part extends circumferentially along the target pipe. The first driving part includes a fixed end and an output end. The fixed end of the first driving part is connected to the clamping part, and the output end of the first driving part is connected to the first driving wheel. The rotation axis of the first driving wheel is perpendicular to the extension direction of the target pipe. The plane on which the first driving wheel is located coincides with the axis of the target pipe. The circumferential surface of the first driving wheel abuts against the pipe wall of the target pipe.

[0044] An image capturing assembly includes a track, a slider, and a capturing unit. The track is connected to the clamping unit, and the track and the clamping unit extend in the same direction. The slider is slidably connected to the track, and the capturing unit is connected to the track. The capturing direction of the capturing unit is towards the wall of the target pipe.

[0045] Furthermore, the clamping part includes multiple fixed clamping plates, two movable clamping plates, and multiple springs. The multiple fixed clamping plates and the two movable clamping plates are parallel to the axis of the target pipe. The multiple fixed clamping plates are connected end-to-end and arranged circumferentially along the target pipe. The two movable clamping plates are rotatably connected to the two fixed clamping plates located at both ends, and their rotation axes are parallel to the target pipe. The two ends of the springs are respectively connected to the fixed clamping plates and the movable clamping plates. There are multiple first driving units and multiple first driving wheels. The output ends of the multiple first driving units are connected one-to-one with the multiple first driving wheels. The fixed ends of the multiple first driving units are respectively connected to the fixed clamping plates and the movable clamping plates on the side facing the target pipe. The multiple first driving wheels are all located between the clamping part and the target pipe.

[0046] Furthermore, the track is flat and perpendicular to the fixed clamp. A groove extending circumferentially along the target pipe is formed on the track. The slider includes a sliding rod, two fixed support plates, two casters, two second drive units, and two second drive wheels. The sliding rod is inserted into the groove and parallel to the target pipe. Both fixed support plates are connected to one end of the sliding rod and are located on opposite sides of the track along the extension direction of the sliding rod, parallel to the plane of the track. The casters are connected to the side of one fixed support plate facing the track and abut against the track. The second drive unit includes a fixed end and an output end. The fixed end of the second drive unit is connected to the side of the other fixed support plate facing the track, and the output end of the second drive unit is connected to the second drive wheel. The axis of the second drive wheel is perpendicular to and passes through the axis of the target pipe, and the circumferential surface of the second drive wheel abuts against the track. The imaging unit is connected to the other end of the sliding rod.

[0047] This invention provides a pipeline quality inspection method based on panoramic vision. It first acquires multiple sets of partial images of the target pipeline wall. Multiple partial images in the same set correspond to the same axial position on the target pipeline, and the shooting directions of these images are arranged around the target pipeline based on their corresponding axial positions. Different sets of partial images correspond to different axial positions. Then, all the partial images in each set are stitched together to obtain multiple partial panoramic images. These multiple partial panoramic images are then stitched together to obtain a panoramic pipeline image. Finally, based on the panoramic pipeline image, a preset target detection model is used to detect and identify target defects. Compared to existing technologies, this invention obtains a panoramic pipeline image by stitching together partial images from multiple angles, and then uses a preset target detection model to complete the detection. Its vision-based detection only requires a camera, is inexpensive, and has wide applicability. Most of the work can be done by computer software. Apart from the camera, there are no requirements for the accuracy of equipment such as laser probes. It is less affected by environmental factors, and the detection results are stable, making it highly practical. Attached Figure Description

[0048] Figure 1 This is a flowchart illustrating an embodiment of the pipeline quality inspection method based on panoramic vision provided by the present invention.

[0049] Figure 2 for Figure 1 A flowchart of a method according to an embodiment of step S102;

[0050] Figure 3 for Figure 2 A flowchart of a method according to an embodiment of step S201;

[0051] Figure 4 A mapping diagram from cylindrical space to planar space in one embodiment of the pipeline quality inspection method based on panoramic vision provided by the present invention;

[0052] Figure 5 for Figure 4 Top view;

[0053] Figure 6 for Figure 4 Side view;

[0054] Figure 7 This is a schematic diagram of an embodiment of the pipeline quality inspection device based on panoramic vision provided by the present invention.

[0055] Figure 8 for Figure 7 A schematic diagram of the structure of the image acquisition module;

[0056] Figure 9 for Figure 8 A structural diagram from another perspective. Detailed Implementation

[0057] Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, which form part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not intended to limit the scope of the present invention.

[0058] In the description of this application, "multiple" means two or more, unless otherwise expressly and specifically defined.

[0059] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0060] This invention obtains a panoramic view of the target pipe surface by stitching together local images from multiple angles. Then, the panoramic view is analyzed using a preset target detection model to identify defects. Most steps can be implemented using computer algorithms, minimizing the use of detection hardware such as probes and greatly reducing the environmental impact that may occur during the quality inspection process.

[0061] This invention provides a pipeline quality inspection method and a pipeline quality inspection device based on panoramic vision, which will be described below.

[0062] Combination Figure 1 As shown, a specific embodiment of the present invention discloses a pipeline quality inspection method based on panoramic vision, comprising:

[0063] S101. Obtain multiple sets of partial images of the target pipe wall. Multiple partial images in the same set correspond to the same axial position on the target pipe. The shooting direction of multiple partial images in the same set is arranged around the target pipe based on the corresponding axial position. The axial positions corresponding to different sets of partial images are different.

[0064] S102. Stitch together all the local images in each group to obtain multiple local panoramic images;

[0065] S103. Stitch together multiple partial panoramic images to obtain a panoramic view of the pipeline;

[0066] S104. Based on the pipeline panoramic image, detect the pipeline panoramic image using a preset target detection model to identify and locate target defects.

[0067] Compared to existing technologies, this invention obtains a panoramic view of the pipeline by stitching together local images from multiple angles, and then completes the detection using a preset target detection model. Its vision-based detection only requires a camera, is inexpensive and widely applicable, and most of the work can be done by computer software. Apart from the camera, there are no requirements for the accuracy of equipment such as laser probes, it is less affected by the environment, and the detection results are stable, making it highly practical.

[0068] Step S101 can be implemented using any existing technology, such as taking pictures of the surface or inner wall of the target pipe using a camera. During the shooting, a set of partial pictures from different angles can be taken around the surface of the pipe, and then another set can be taken by moving along the extension direction of the pipe.

[0069] Combination Figure 2 As shown, in a preferred embodiment, step S102, stitching together all the local images in each group to obtain multiple local panoramic images, specifically includes...

[0070] S201. Preprocess each of the local images in the same group to obtain multiple optimized images;

[0071] S202. Based on the actual positional relationship of the optimization maps, sort the multiple optimization maps in the same group;

[0072] S203. In the same group, the i-th stitched image and the j-th optimized image are stitched together to obtain the (i+1)-th stitched image;

[0073] Wherein, i is the number of stitching, j is the sorting number of the multiple optimized images, i and j are both positive integers, and j = i + 2. The first stitched image is formed by stitching together the first optimized image and the second optimized image, and the final stitched image is the local panoramic image.

[0074] The above process is to stitch a set of images into a partial panoramic image, that is, to stitch multiple partial images from a set into an unfolded image of a certain segment of the target pipeline.

[0075] Specifically, in combination Figure 3 As shown, in a preferred embodiment, step S201, which involves preprocessing each of the local images in the same group to obtain multiple optimized images, specifically includes:

[0076] S301. Denoise the local image;

[0077] S302. Extract the pipe image from the denoised local image to obtain a local pipe image;

[0078] S303. Perform cylindrical back projection transformation on the local piping diagram to obtain the piping unfolded diagram;

[0079] S304. Repair and optimize the pipeline unfolded diagram to obtain the optimized diagram.

[0080] The present invention also provides a more detailed embodiment to more clearly illustrate the above steps S301 to S304:

[0081] In this embodiment, the image preprocessing operation refers to first using Gaussian filtering to denoise the image; then, by specifying the range of rows or columns of interest, the ROI region is cropped to extract the outer pipe surface portion of the image (i.e., obtaining the local pipe map); next, cylindrical back projection transformation is performed on the image to unfold the cylinders on the outer surface of the pipe (i.e., obtaining the unfolded pipe map); finally, a Navier-Stokes-based repair algorithm is used to repair the missing pixels in the unfolded image, obtaining a complete optimized image of the cylinder unfolding. It is understood that other existing methods can also be used for repair in practice.

[0082] Specifically, the cylindrical back-projection transformation in the above process refers to a generalization based on the principle of cylindrical projection, which involves reversing the original and target surfaces to project the cylindrical image onto a planar image. The mapping relationship from cylindrical space to planar space during the cylindrical back-projection transformation is as follows: Figure 4 As shown. The curved surface JDILCK is the original image to be processed, which is transformed into a rectangle GHEF after being projected onto the plane. By analyzing its top view, the x-coordinates of the pixels on the cylindrical and plane surfaces can be obtained (i.e., x... ′ The projection relationship between x and x is: Where f is the distance from point B to rectangle GHEF, i.e. Figure 5 As shown. The mapping relationship between the pixels of the cylinder and the projection plane in the Y direction is determined by the side view. The line segment can be obtained from the top view. θ is the x in the graph ′ The angle between B and DB can be obtained from the similar triangle relationship in the side view, which gives the ordinate (y) of the pixel point on the cylinder and plane. ′ The projection relationship between y and y is: like Figure 6 As shown. In summary, the correspondence between the coordinates of each pixel on the original image and the projection plane can be obtained as follows:

[0083]

[0084]

[0085]

[0086] On the other hand, the Navier-Stokes-based restoration algorithm in the above process is based on fluid dynamics and utilizes partial differential equations. It maps the stream function X in two-dimensional incompressible fluid dynamics to the grayscale function I of the image to be restored in image restoration, where the fluid velocity... And the direction of isolux lines Correspondingly, the vorticity function ω = Δψ and the Laplacian smoothness ω = ΔI correspond, as do the fluid viscosity coefficient v and the heterosmoothness coefficient v. The Navier-Stokes equations are listed below:

[0087]

[0088] The function g is a monotonically decreasing function, causing the function ω to diffuse heterogeneously. This is achieved by solving the Poisson equation:

[0089]

[0090] Finally, we obtain the optimized diagram I after repair.

[0091] Furthermore, in a preferred embodiment, step S202, which involves sorting multiple optimization maps in the same group based on their actual positional relationships, specifically includes:

[0092] Based on the axial position corresponding to each group of optimization diagrams, the multiple groups of optimization diagrams are sorted among themselves.

[0093] Based on the surrounding order of the shooting direction corresponding to each of the optimized images in the same group, the optimized images in the kth group are sorted in ascending order;

[0094] Based on the surrounding order of the shooting direction corresponding to each of the optimized images in the same group, the optimized images in the (k+1)th group are sorted in reverse order;

[0095] Wherein, k refers to the number of times each group of the optimized graphs is passed through.

[0096] It is understandable that, since the optimized images are obtained from the preprocessing of local images, there is a one-to-one correspondence between the two. Therefore, the axial position and shooting angle of the optimized image in the above process can be referenced from the axial position and shooting angle of its corresponding local image. Specifically, the above process means that two adjacent sets of optimized images are sorted alternately according to the shooting position relationship of each image. For example, a numbered index is created for each group according to the order of the axial position along the pipe extension. The images in even-numbered index groups and the sorting between groups (starting from group 0) are sorted according to the shooting time order, while the images in odd-numbered index groups are sorted in reverse order of shooting time.

[0097] Furthermore, in a preferred embodiment, step S203, which involves stitching the i-th stitched image and the j-th optimized image within the same group to obtain the (i+1)-th stitched image, specifically includes:

[0098] Extract feature descriptors from the i-th stitched image and the j-th optimized image;

[0099] Based on the feature descriptor, the perspective transformation matrix is ​​obtained;

[0100] According to the perspective transformation matrix, the i-th stitched image and / or the j-th optimized image are subjected to perspective transformation to obtain two images to be merged.

[0101] The optimal suture line is obtained based on the overlapping area of ​​the two images to be fused.

[0102] Based on the optimal suture line, the two images to be fused are fused together to obtain the (i+1)th spliced ​​image.

[0103] Wherein, i is the number of stitching, j is the sorting number of the multiple optimized images, i and j are both positive integers, and j = i + 2. The first stitched image is formed by stitching together the first optimized image and the second optimized image, and the final stitched image is the local panoramic image.

[0104] The present invention also provides a more detailed embodiment to more clearly illustrate step S203 above:

[0105] In this embodiment, the specific process of stitching together images in the same group is as follows: the first two images in the group are image registered and stitched together to form a new image. Then, the new image is image registered and stitched together with the next image in the same group, and so on, until all images in the group are stitched together to form a completely new image.

[0106] Specifically, the image registration process described above involves first extracting feature points from each image using the SIFT algorithm and calculating the feature vectors of the regions surrounding each feature point. Then, using the feature points and feature vectors of the two images (i.e., the feature descriptors; it's understood that other existing image registration methods can also be used, and the parameters required by other methods can also be considered as feature descriptors), a perspective transformation matrix M is obtained through FLANN homography matching. Finally, the obtained perspective transformation matrix M is used to perform a perspective transformation on the second image, making the perspective of the second image consistent with that of the first image (the two images after perspective adjustment are the two images to be fused). Similarly, other existing methods can also be used in practice to obtain the perspective transformation matrix or to adjust the perspective.

[0107] The SIFT algorithm in the above steps refers to extracting feature points that are not affected by factors such as lighting, scale, and rotation through the Gaussian differential function, determining the position and scale information of each selected feature point through the fitted model, comparing the feature points of two images, and establishing the correspondence between objects in the two images.

[0108] The FLANN feature detection in the above steps refers to calculating, comparing, and filtering the feature descriptors of two images to select feature points with high matching degree between the two images, thereby obtaining the perspective transformation matrix M that changes between the selected matching points.

[0109] Furthermore, the stitching and fusion process described above refers to first using dynamic programming to find the seam line in the overlapping area of ​​two images that is most similar in color and structure, i.e., the optimal seam line; and then using a Laplacian pyramid algorithm based on a mask and the optimal seam line to achieve a smooth transition at the image fusion point.

[0110] The optimal suture line algorithm in the above steps refers to finding a line with the minimum cumulative strength value. The calculation of the optimal suture line strength value is shown in the following formula:

[0111] E(x, y) = E C (x, y) 2 +E g (x, y)

[0112] Among them, E C E is represented as the difference in color values ​​between overlapping pixels. g It is represented as the difference in structural values ​​of overlapping pixels.

[0113] The Laplacian pyramid algorithm based on mask and optimal stitching line in the above steps refers to creating a mask image for the fused image based on the optimal stitching line. The value of each pixel within the fusion region is obtained by linearly fusing the foreground image, background image, and mask image. The fusion formula is as follows:

[0114]

[0115] Among them, I l (x, y) represents the left image to be stitched together, I r (x, y) represents the right image to be stitched together, I p (x, y) represents the merged image, w represents the mask image, and Ω represents the overlapping region. Similarly, it can be understood that the specific methods for obtaining the optimal seam line and image stitching in the above process can also be implemented using other existing methods.

[0116] Furthermore, in a preferred embodiment, step S103, stitching together multiple local panoramic images to obtain a pipeline panoramic image, is the same as step S102. That is, each local panoramic image is stitched together as a local image from step S102, and the final image obtained is the pipeline panoramic image. In this embodiment, each local panoramic image needs to be preliminarily adjusted before stitching. Specifically, each obtained local panoramic image is rotated 90 degrees counterclockwise to form a new set of images, in which the height and width of the image bands are interchanged. The counterclockwise rotation in this step refers to first reversing the image matrix row by row, and then transposing it. Then, the flipped images are stitched together, and then rotated 90 degrees clockwise to obtain the pipeline panoramic image. The clockwise rotation in this step refers to first transposing the image matrix, and then reversing it row by row.

[0117] Furthermore, in a preferred embodiment, step S104, detecting the pipeline panorama based on a preset target detection model and identifying and locating target defects according to the pipeline panorama, specifically includes:

[0118] Obtain a sample dataset characterizing pipeline defects;

[0119] A neural network model is established, and the neural network model is trained based on the sample dataset to obtain the preset target detection model;

[0120] The pipeline panoramic image is detected based on the preset target detection model to identify and locate target defects.

[0121] The above process can identify the type of defect and its location on the pipeline using a panoramic view of the pipeline. This process can be implemented using any existing image recognition neural network model. For example, in this embodiment, the process can be specifically divided into the following seven steps:

[0122] Data collection involved creating a dataset by taking photos of welds, pipes, etc., obtained from online searches and real-life observations. The data was then randomly divided into training, testing, and validation sets at ratios of 0.8, 0.1, and 0.1, and labeled with two tags: "good" and "bad" (i.e., obtaining sample datasets).

[0123] Data reading and preprocessing: Applying appropriate preprocessing methods to the training set can help achieve better convergence and prevent overfitting.

[0124] Convolutional neural networks are used to extract features. MobileNetv3 is used as the backbone network. The output data of C0, C1, and C2 are extracted, and multi-scale detection is used to obtain features (i.e., to build a neural network model).

[0125] Generate candidate regions and annotations. Generate a series of candidate regions on the image and annotate whether they contain objects, the category of the objects, and the adjustment range of the predicted bounding box relative to the anchor box.

[0126] The model is trained by associating the feature maps of the three levels with the labels between the corresponding anchor boxes, establishing a loss function, and starting the end-to-end training process.

[0127] The model is evaluated and tested by calculating the scores of the predicted box location and its category through the network output. Non-maximum suppression is used to eliminate predicted boxes with large overlap, and the final test accuracy is obtained.

[0128] Model deployment: If the model evaluation test results reach a certain accuracy rate, the model is deployed, and the preset target detection model is finally obtained.

[0129] To better implement the pipeline quality inspection method based on panoramic vision in the embodiments of the present invention, based on the pipeline quality inspection method based on panoramic vision, please refer to the corresponding... Figure 7 , Figure 7 This is a schematic diagram of an embodiment of the pipeline quality inspection device based on panoramic vision provided by the present invention. The pipeline quality inspection device 700 based on panoramic vision provided by this embodiment includes:

[0130] The image acquisition module 710 is used to acquire multiple sets of partial images of the wall of the target pipe 900. Multiple partial images in the same set correspond to the same axial position on the target pipe 900, and the shooting direction of multiple partial images in the same set is arranged around the target pipe 900 based on the corresponding axial position. The axial positions corresponding to different sets of partial images are different.

[0131] The primary stitching module 720 is used to stitch together all the local images in each group to obtain multiple local panoramic images;

[0132] The secondary stitching module 730 is used to stitch together multiple partial panoramic images to obtain a panoramic image of the pipeline.

[0133] The defect detection module 740 is used to detect the pipeline panoramic image based on a preset target detection model, and identify and locate target defects.

[0134] It should be noted that the corresponding devices provided in the above embodiments can implement the technical solutions described in the above method embodiments. The specific implementation principles of the above modules or units can be found in the corresponding content in the above method embodiments, and will not be repeated here.

[0135] Furthermore, in combination Figure 8 and Figure 9As shown, in a preferred embodiment, the image acquisition module includes:

[0136] The pipe climbing assembly 711 includes a clamping part 7111, a first driving part 7112, and a first driving wheel 7113. The clamping part 7111 extends circumferentially along the target pipe 900. The first driving part 7112 includes a fixed end and an output end. The fixed end of the first driving part 7112 is connected to the clamping part 7111, and the output end of the first driving part 7112 is connected to the first driving wheel 7113. The rotation axis of the first driving wheel 7113 is perpendicular to the extension direction of the target pipe 900. The plane on which the first driving wheel 7113 is located coincides with the axis of the target pipe 900. The circumferential surface of the first driving wheel 7113 abuts against the pipe wall of the target pipe 900.

[0137] Image capturing assembly 712 includes a track 7121, a slider 7122, and a capturing part 7123. The track 7121 is connected to the clamping part 7111, and the track 7121 and the clamping part 7111 extend in the same direction. The slider 7122 is slidably connected to the track 7121, and the capturing part 7123 is connected to the track 7121. The capturing direction of the capturing part 7123 is towards the wall of the target pipe 900.

[0138] In the above structure, the pipe climbing component 711 is fixed to the target pipe 900 by the clamping part 7111, and then the first drive wheel 7113 is driven by the first drive part 7112 to complete the movement. In the image capturing component 712, the capturing part 7123 moves on the track 7121 by the slider 7122 to complete the movement around the target pipe 900, thereby realizing the acquisition of local images from multiple angles.

[0139] Specifically, in a preferred embodiment, the clamping part 7111 includes a plurality of fixed clamping plates 71111, two movable clamping plates 71112, and a plurality of springs 71113. The plurality of fixed clamping plates 71111 and the two movable clamping plates 71112 are all parallel to the axis of the target pipe 900. The plurality of fixed clamping plates 71111 are connected end-to-end and arranged circumferentially along the target pipe 900. The two movable clamping plates 71112 are rotatably connected to the two fixed clamping plates 71111 located at both ends, and their rotation axes are parallel to the target pipe 900. The springs 71113... The two ends of 113 are respectively connected to the fixed clamping plate 71111 and the movable clamping plate 71112; there are multiple first driving parts 7112 and multiple first driving wheels 7113, and the output ends of multiple first driving parts 7112 are connected to multiple first driving wheels 7113 in a one-to-one correspondence. The fixed ends of multiple first driving parts 7112 are respectively connected to the side of the fixed clamping plate 71111 and the movable clamping plate 71112 facing the target pipe 900. The multiple first driving wheels 7113 are all located between the clamping part 7111 and the target pipe 900.

[0140] Furthermore, in a preferred embodiment, the track 7121 is flat and perpendicular to the fixed clamping plate 71111. A groove extending circumferentially along the target pipe 900 is formed on the track 7121. The slider 7122 includes a sliding rod 71221, two fixed support plates 71222, two casters 71223, two second drive units 71224, and two second drive wheels 71225. The sliding rod 71221 is inserted into the groove and is parallel to the target pipe 900. Both fixed support plates 71222 are connected to one end of the sliding rod 71221, and are located on the track 7121 along the sliding rod 71221. The universal wheel 71223 is connected to one side of the fixed support plate 71222 facing the track 7121 and abutting against the track 7121. The second drive unit 71224 includes a fixed end and an output end. The fixed end of the second drive unit 71224 is connected to the other side of the fixed support plate 71222 facing the track 7121. The output end of the second drive unit 71224 is connected to the second drive wheel 71225. The axis of the second drive wheel 71225 is perpendicular to and passes through the axis of the target pipe 900. The circumferential surface of the second drive wheel 71225 abuts against the track 7121. The shooting unit 7123 is connected to the other end of the sliding rod 71221.

[0141] It is understandable that the aforementioned track 7121 and slider 7122 can also be implemented using any other sliding structure. The imaging unit 7123 is a camera.

[0142] The present invention also provides a more specific embodiment to illustrate the above-described image acquisition module:

[0143] The pipeline quality inspection device based on panoramic vision in this embodiment can be referred to as a robot. The first drive unit 7112, which is set on the inner wall of the fixed clamping plate 71111 and the movable clamping plate 71112, is a 12V geared motor. The universal wheel 71223 is a bullseye wheel. The second drive unit 71224 is also a motor. There are two of each of the first drive unit 7112 and the second drive wheel 71225. By controlling the forward and reverse rotation of the first drive unit 7112, the robot's forward and backward movement can be controlled. By controlling the speed difference between the two motors of the second drive unit 71224, the imaging unit 7123 can move on the track 7121, so as to achieve 360° no-blind-angle imaging around the pipeline when the robot crawls.

[0144] The main body of the robot shell consists of six surfaces (i.e., four fixed clamping plates 71111 and two movable clamping plates 71112). The angle between the four surfaces is fixed and they share a common inscribed circle. The diameter of this inscribed circle determines the maximum outer diameter of the pipe that the robot can clamp. The other two surfaces act as clamping mechanisms in conjunction with springs 71113 to achieve tension. The compression of springs 71113 provides clamping force, allowing the crawling motion mechanism to adhere tightly to the outer wall of the pipe.

[0145] In this embodiment, the robot is driven forward or backward by a motor, and can adjust its speed and stop abruptly. The robot moves forward or backward a set distance each time and hovers until the camera takes a picture before continuing its next movement. The robot has two modes: automatic operation and manual infrared remote control. Details are as follows:

[0146] Taking the movement of a robot on a pipe as an example, the methods for forward and backward movement, speed adjustment, and emergency stop are as follows:

[0147] Before movement, the robot is firmly fixed to the pipe. The robot's state is changed by increasing the motor's driving force. Simulations can be used to calculate the critical motor driving forces for forward and backward movement. When the motor rotates forward and reaches the critical driving force for forward movement, the robot begins to move. At this point, the motor's driving force is stabilized, and the robot moves forward at a stable speed. When the motor rotates in reverse and reaches the critical driving force for backward movement, the robot begins to move. At this point, the motor's driving force is stabilized, and the robot moves backward at a stable speed.

[0148] The speed of a motor is controlled by adjusting the duty cycle using PWM (Pulse Width Modulation). The duty cycle is the ratio of high voltage levels within one cycle; the larger the ratio of high voltage levels, the larger the duty cycle. A DC motor will rotate when the output pin is at a high level. When the output pin is high, the motor will rotate. If the output level suddenly changes from high to low, the motor will not stop due to the inductor's ability to prevent sudden current changes; it will maintain its original speed. The motor speed is the average output voltage value within one cycle, and the average speed over one cycle is the speed determined by the duty cycle. In motor control, the higher the voltage, the faster the motor speed. By outputting different analog voltages using PWM, different output speeds can be achieved.

[0149] When the robot needs to stop urgently, the driving force of the motor is quickly reduced to the critical driving force through PWM adjustment, and then reduced to zero, so that the robot stops quickly and remains stationary.

[0150] In the image capturing assembly 712, a second drive wheel 71225 and a caster wheel 71223 are respectively mounted on the inner sides of two steel plates (i.e., the fixed support plate 71222), which serve to move and clamp. The camera is fixed on the sliding rod 71221, which is connected to the steel plate. The caster wheel 71223, as a driven part, is separately mounted on both sides of the track 7121 plate along with the second drive wheel 71225 to assist in movement.

[0151] The second drive wheel 71225 acts as the active wheel, driving the camera to reciprocate along the track 7121. This allows for circumferential image acquisition of the pipe wall surface using the camera's panoramic view. It's understood that the track 7121 does not need to circle the target pipe 900; it only needs to ensure the camera's field of view covers the entire pipe wall. The specific principle of the camera's movement is as follows:

[0152] Taking the robot's movement on the pipe wall as an example, the camera simultaneously performs a cyclical reciprocating motion. The principle is as follows:

[0153] When starting counterclockwise, the motor's movement drives the second drive wheel 71225 to move along the circular track 7121. Due to the circular motion, the input signals of the two drive motors are different, controlling the duty cycle of the inner drive motor to be less than that of the outer drive motor. This makes the wheel speed of the inner second drive wheel 71225 less than that of the outer second drive wheel 71225, facilitating the camera's movement along the arc track 7121, reducing wear on the support column, and reducing movement resistance.

[0154] Before the camera moves counterclockwise to the end of the curved track 7121, the motor brings both wheels to a sudden stop. Then the motor reverses, and the camera moves clockwise along track 7121, stopping abruptly before the end of track 7121. Then the motor reverses again, and the camera moves counterclockwise, starting the next reciprocating cycle.

[0155] The camera takes photos at fixed times and locations during its clockwise or counterclockwise movement. During this movement, the second drive wheel 71225 propels the camera. When the second drive wheel 71225 is stationary, the camera takes a picture. By adjusting the timing of the output signal sent to the motor and the time without an output signal, as well as utilizing friction, the camera is made to stop at relatively regular positions for taking pictures.

[0156] On the other hand, the primary stitching module 720, secondary stitching module 730, and defect detection module 740 in this embodiment can all be integrated into a computer, host computer, or mobile phone, and implemented through APP, software, etc. For example, in this embodiment, the movement of the robot can be remotely controlled by a mobile phone through local area network communication, and the image can be transmitted back in real time by the APP on the mobile phone. The angle covered by the camera is calculated by calculating the distance between the camera and the surface of the object being photographed. When the angle turned by the moving part of the camera is equal to 3 / 2 times the angle of the photograph coverage, the moving part of the camera temporarily stops moving. After stopping and stabilizing, the APP takes a picture, and the picture is stored in the memory card. After storage, the moving part of the camera resumes movement, and the above process is repeated.

[0157] Meanwhile, the robot takes photos in real time and transmits the data back to a mobile app via 5G. The mobile app performs image preprocessing, image registration, and image stitching and fusion to stitch the images together. Then, it uses a pre-deployed target detection model to detect defects. Finally, the results are sent back to the computer on the mobile phone, where a trained algorithm is used to identify the defects and the computer receives the identification conclusion.

[0158] The structure of this invention is simple, the cost is low, the identification is stable and reliable, and it can solve problems such as high-altitude pipeline detection.

[0159] This invention provides a pipeline quality inspection method based on panoramic vision. It first acquires multiple sets of partial images of the target pipeline wall. Multiple partial images in the same set correspond to the same axial position on the target pipeline, and the shooting directions of these images are arranged around the target pipeline based on their corresponding axial positions. Different sets of partial images correspond to different axial positions. Then, all the partial images in each set are stitched together to obtain multiple partial panoramic images. These multiple partial panoramic images are then stitched together to obtain a panoramic pipeline image. Finally, based on the panoramic pipeline image, a preset target detection model is used to detect and identify target defects. Compared to existing technologies, this invention obtains a panoramic pipeline image by stitching together partial images from multiple angles, and then uses a preset target detection model to complete the detection. Its vision-based detection only requires a camera, is inexpensive, and has wide applicability. Most of the work can be done by computer software. Apart from the camera, there are no requirements for the accuracy of equipment such as laser probes. It is less affected by environmental factors, and the detection results are stable, making it highly practical.

[0160] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A pipeline quality inspection method based on panoramic vision, characterized in that, include: Multiple sets of partial images of the target pipe wall are acquired. The multiple partial images in the same set correspond to the same axial position on the target pipe, and the shooting direction of the multiple partial images in the same set is arranged around the target pipe based on the corresponding axial position. The axial positions corresponding to different sets of partial images are different. By stitching together all the local images in each group, multiple local panoramic images are obtained; By stitching together multiple partial panoramic images, a panoramic image of the pipeline is obtained; Based on the pipeline panoramic image, the pipeline panoramic image is detected using a preset target detection model to identify and locate target defects. By stitching together all the partial images in each group, multiple partial panoramic images are obtained, including: Preprocessing is performed on each of the local images in the same group to obtain multiple optimized images, including: denoising the local images; extracting pipe images from the denoised local images to obtain local pipe images; performing cylindrical back projection transformation on the local pipe images to obtain pipe unfolded images; and repairing and optimizing the pipe unfolded images to obtain the optimized images. Based on the actual positional relationship of the optimized images, the multiple optimized images in the same group are sorted, including: sorting the multiple groups of optimized images between groups based on the axial position corresponding to each group of optimized images; sorting the optimized images in the k-th group in ascending order based on the surround order of the shooting direction corresponding to each optimized image in the same group; and sorting the optimized images in the (k+1)-th group in reverse order based on the surround order of the shooting direction corresponding to each optimized image in the same group; wherein, k refers to each group of optimized images. In the same group, the i-th stitched image and the j-th optimized image are stitched together to obtain the (i+1)-th stitched image; Wherein, i is the number of stitching, j is the sorting number of the multiple optimized images, i and j are both positive integers, and j = i + 2. The first stitched image is formed by stitching together the first optimized image and the second optimized image, and the final stitched image is the local panoramic image. In the same group, the i-th stitched image and the j-th optimized image are stitched together to obtain the (i+1)-th stitched image, including: Extract feature descriptors from the i-th stitched image and the j-th optimized image; Based on the feature descriptor, the perspective transformation matrix is ​​obtained; According to the perspective transformation matrix, the i-th stitched image and / or the j-th optimized image are subjected to perspective transformation to obtain two images to be merged. The optimal suture line is obtained based on the overlapping area of ​​the two images to be fused. Based on the optimal suture line, the two images to be fused are fused together to obtain the (i+1)th spliced ​​image.

2. The pipeline quality inspection method based on panoramic vision according to claim 1, characterized in that, The step of detecting the pipeline panoramic image based on a preset target detection model, and identifying and locating target defects, includes: Obtain a sample dataset characterizing pipeline defects; A neural network model is established, and the neural network model is trained based on the sample dataset to obtain the preset target detection model; The pipeline panoramic image is detected based on the preset target detection model to identify and locate the target defect.

3. A pipeline quality inspection device based on panoramic vision, applied to the pipeline quality inspection method based on panoramic vision as described in claim 1 or 2, characterized in that, The pipeline quality inspection device based on panoramic vision includes: The image acquisition module is used to acquire multiple sets of partial images of the target pipe wall. Multiple partial images in the same set correspond to the same axial position on the target pipe, and the shooting direction of multiple partial images in the same set is arranged around the target pipe based on the corresponding axial position. The axial positions corresponding to different sets of partial images are different. A primary stitching module is used to preprocess each of the local images in the same group to obtain multiple optimized images, including: denoising the local images; extracting pipe images from the denoised local images to obtain local pipe maps; performing cylindrical back projection transformation on the local pipe maps to obtain pipe unfolded maps; repairing and optimizing the pipe unfolded maps to obtain the optimized maps; sorting the multiple optimized maps in the same group based on the actual positional relationship of the optimized maps, including: sorting the multiple groups of optimized maps between groups based on the axial position corresponding to each group of optimized maps; sorting the optimized maps in the k-th group in ascending order based on the surround order of the shooting direction corresponding to each optimized map in the same group; sorting the optimized maps in the k+1-th group in reverse order based on the surround order of the shooting direction corresponding to each optimized map in the same group; wherein, k refers to each group of optimized maps; stitching together all the local images in each group to obtain multiple local panoramic images; The secondary stitching module is used to stitch together multiple partial panoramic images to obtain a panoramic image of the pipeline. The defect detection module is used to detect the pipeline panoramic image based on a preset target detection model, and identify and locate target defects.

4. The pipeline quality inspection device based on panoramic vision according to claim 3, characterized in that, The image acquisition module includes: A pipe climbing assembly includes a clamping part, a first driving part, and a first driving wheel. The clamping part extends circumferentially along the target pipe. The first driving part includes a fixed end and an output end. The fixed end of the first driving part is connected to the clamping part, and the output end of the first driving part is connected to the first driving wheel. The rotation axis of the first driving wheel is perpendicular to the extension direction of the target pipe. The plane on which the first driving wheel is located coincides with the axis of the target pipe. The circumferential surface of the first driving wheel abuts against the pipe wall of the target pipe. An image capturing assembly includes a track, a slider, and a capturing unit. The track is connected to the clamping unit, and the track and the clamping unit extend in the same direction. The slider is slidably connected to the track, and the capturing unit is connected to the track. The capturing direction of the capturing unit is towards the wall of the target pipe.

5. The pipeline quality inspection device based on panoramic vision according to claim 4, characterized in that, The clamping part includes multiple fixed clamping plates, two movable clamping plates, and multiple springs. The fixed clamping plates and the two movable clamping plates are parallel to the axis of the target pipe. The fixed clamping plates are connected end-to-end and arranged circumferentially along the target pipe. The two movable clamping plates are rotatably connected to the two fixed clamping plates located at both ends, with their rotation axes parallel to the target pipe. The two ends of the springs are connected to the fixed clamping plates and the movable clamping plates, respectively. There are multiple first driving units and multiple first driving wheels. The output ends of the multiple first driving units are connected one-to-one with the multiple first driving wheels. The fixed ends of the multiple first driving units are connected to the fixed clamping plates and the movable clamping plates on the side facing the target pipe, respectively. The multiple first driving wheels are located between the clamping part and the target pipe.

6. The pipeline quality inspection device based on panoramic vision according to claim 5, characterized in that, The track is flat and perpendicular to the fixed clamp. A groove extending circumferentially along the target pipe is formed on the track. The slider includes a sliding rod, two fixed support plates, two casters, two second drive units, and two second drive wheels. The sliding rod is inserted into the groove and parallel to the target pipe. Both fixed support plates are connected to one end of the sliding rod and are located on opposite sides of the track along the extension direction of the sliding rod, parallel to the plane of the track. The casters are connected to the side of one fixed support plate facing the track and abut against the track. The second drive unit includes a fixed end and an output end. The fixed end of the second drive unit is connected to the side of the other fixed support plate facing the track, and the output end of the second drive unit is connected to the second drive wheel. The axis of the second drive wheel is perpendicular to and passes through the axis of the target pipe, and the circumferential surface of the second drive wheel abuts against the track. The imaging unit is connected to the other end of the sliding rod.

Images