A pipeline image processing method and imaging device
By using multiple laser emitters and cameras for image correction in municipal pipeline inspection, combined with target detection algorithms and ranging modules, the problems of image distortion and silt identification in pipeline inspection were solved, achieving accurate defect assessment and silt judgment.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- GUANGDONG TECHN COLLEGE OF WATER RESOURCES & ELECTRIC ENG
- Filing Date
- 2026-04-21
- Publication Date
- 2026-07-31
AI Technical Summary
In existing technologies, when municipal pipeline inspection robots use panoramic cameras to capture images, the images are distorted, making it difficult to accurately assess the number and size of defects, and there is a lack of effective means to identify and quantify silt deposits.
Multiple sets of laser emitters and cameras are used to perform image rotation and scaling correction using laser points as a reference. Defects are identified by combining target detection algorithms, and a three-dimensional model of the pipeline is constructed using four cross-shaped ranging modules to determine whether silt is present.
It achieves continuous and seamless 2D panoramic image stitching, significantly reducing the risk of missed or incorrect detection, accurately identifying defects, accurately assessing the structural status of pipelines, and visualizing the sludge surface line in the 3D model, providing a basis for dredging decisions.
Smart Images
Figure CN122492439A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of panoramic imaging technology, and more specifically to a pipeline image processing method and imaging device. Background Technology
[0002] Municipal pipelines are typically circular and buried beneath municipal roads. With increasing service life, these pipelines are prone to structural defects such as cracks, deformation, corrosion, misalignment, unevenness, disconnection, joint material detachment, concealed branch connections, foreign object penetration, and leakage, as well as functional defects such as sedimentation, scaling, obstructions, residual wall / dam roots, tree roots, and scum, due to long-term exposure to internal fluid erosion and external factors like soil pressure and groundwater level fluctuations. Therefore, regular internal inspections are necessary. A common method is to use a detection robot to enter the pipeline and photograph the inner wall to determine the presence of defects. Currently, when robots use panoramic cameras to photograph the inner wall of municipal pipelines, the images are taken from the side, resulting in distortion. This distortion causes cracks and other defects to appear deformed, making it difficult to accurately assess the number and size of defects. Furthermore, existing detection systems lack effective identification and quantitative analysis methods for sludge deposition within pipelines, failing to meet the needs of assessing sedimentation conditions. Summary of the Invention
[0003] In view of the technical problems existing in the prior art, the purpose of this invention is to provide a pipeline image processing method and imaging device, which can obtain a continuous, seamless and clear two-dimensional panoramic image, thereby enabling more accurate and realistic identification of defects and assessment of pipeline structural condition.
[0004] To achieve the above objectives, the present invention adopts the following technical solution: a pipeline image processing method, comprising the following steps,
[0005] S1. Multiple laser emitters and multiple cameras are set on the imaging device. Multiple laser points are emitted along the circumferential direction of the inner wall of the pipe by the laser emitters. Each set of laser points includes two laser points. Multiple images are captured along the circumferential direction of the inner wall of the pipe by the cameras, and it is ensured that there is a set of the same laser points in every two adjacent images. Thus, there are two sets of laser points in each image.
[0006] S2, each original image captured is recorded as image A. Each image A is rotated and scaled so that the line segment formed by the same set of two laser points is rotated to the horizontal direction and the line segment length is consistent in the transformed image, thus obtaining image B. The rotation angle value, scaling ratio value and coordinate values of the four laser points in image B are saved for each image B.
[0007] S3, turn off the laser emitter, and the camera takes another image of the inner wall of the pipe, which is recorded as image C;
[0008] S4. Based on the rotation angle value and scaling ratio value obtained in step S2, perform the same rotation and scaling transformation on the corresponding image C to obtain image D;
[0009] S5. Based on the coordinates of the four laser points obtained in step S2, locate the corresponding four points in image D, use these four points as vertices to extract a quadrilateral region, and obtain image E.
[0010] S6. Stitch the images E sequentially to obtain a two-dimensional pipe stitched panoramic image, denoted as image G.
[0011] As a preferred approach, a target detection algorithm is used to identify defects in image G, thereby obtaining the number and size of defects.
[0012] As a preferred option, if no defects cross the last image E and the first image E in the pipeline stitched panoramic image, the defect identification step ends;
[0013] If a defect exists that spans between the last image E and the first image E, then perform the following steps:
[0014] S7.1, with the length representing the circumferential direction of the pipe as the ordinate and the axial direction of the pipe as the abscissa, find a horizontal line with the ordinate y1 in the image G, requiring that no defects cross this horizontal line.
[0015] If a horizontal line that meets the requirements exists in image G, then proceed to step S7.2; otherwise, proceed to steps S7.3 to S7.6.
[0016] S7.2, divide the image G into two images G1 and G2 by the horizontal line with the vertical coordinate y1, stitch G2 and G1 together in order to form an image H, and make the lower edge of G2 connect with the upper edge of G1 to obtain image H;
[0017] S7.3, the target detection algorithm identifies defects in image G from top to bottom. The lowest ordinate of the first complete defect is y2. The horizontal line with the ordinate y2 divides image G into two images, G3 and G4.
[0018] S7.4 In image G, starting from the second complete defect, find a defect that passes through the horizontal line y2 and has the highest top. The top of this defect is marked with the vertical coordinate y3. Divide image G3 into two images G31 and G32 by the horizontal line with the vertical coordinate y3.
[0019] S7.5, stitch together G32, G4, and G3 in the top-to-bottom order to form an image I;
[0020] S7.6 Mark the repeated incomplete defects in area G32 of image I in red, and mark the remaining defects in green.
[0021] As a preferred method, the method for determining whether a defect spans the last image and the first image is as follows: the last image E and the first image E are stitched together to form an image F. The location and size values of the defect are identified using an object detection algorithm. Combined with the sizes of the two images, it is determined whether a defect spans the two images.
[0022] As a preferred option, the following steps are also included:
[0023] S8. An angle sensor and four ranging modules are set on the imaging device. The ranging directions of the four ranging modules are distributed in a cross shape within the same pipe cross section. The pipe radius R is calculated based on the distance value measured by the ranging modules. Then, based on the radius R and the width W of the image E, a three-dimensional panoramic model of the pipe with radius R and axial length W is constructed.
[0024] S9, based on the tilt angle of the laser emitter, calculate the coordinates of each laser point on the cross-section of the three-dimensional panoramic model of the pipeline to determine the start and end positions of each image E in the circumferential direction of the pipeline.
[0025] S10, geometrically correct image E according to the circumference arc length of the pipeline, and map the corrected image to the corresponding curved surface position of the 3D panoramic model to generate a 3D panoramic image of the pipeline.
[0026] As a preferred method, the pipe radius is calculated using data obtained from the ranging module, and the presence of silt at the bottom of the pipe is determined. The specific process is as follows.
[0027] S8.1, the four ranging modules are arranged in the circumferential direction as the first ranging module, the second ranging module, the third ranging module, and the fourth ranging module. The measured distances are denoted as L1, L2, L3, and L4. The second ranging module measures distances directly downwards. The center point around which the four ranging modules are arranged is denoted as C, and the distance from the four ranging modules to the center point is denoted as r. The ranging points of the first, second, third, and fourth ranging modules inside the pipe are denoted as M, F, N, and D, respectively. Then, we have MC = L1 + r, FC = L2 + r, NC = L3 + r, and DC = L4 + r.
[0028] S8.2, the point on the inner wall of the pipe where the second ranging module extends along the ranging direction is denoted as E. According to the power theorem, ;
[0029] S8.3, obtain the pipe radius R, ;
[0030] When EC is greater than FC, it is determined that there is silt at the bottom of the pipeline. At this time, the measuring point F is the surface of the silt. When EC equals FC, there is no silt at the bottom of the pipeline. At this time, the measuring point F coincides with E.
[0031] As a preferred option, when there is silt at the bottom of the pipe, the silt height and silt height ratio are calculated. The specific process is as follows:
[0032] S11.1, the line on the surface of the silt at the bottom of the pipe is denoted as line segment AB, and the center point of the pipe is denoted as O; the perpendicular line from O to line segment MN is denoted as H, and the intersection of the extension of line segment OH and line segment AB is denoted as I; the perpendicular line from O to line segment DE is denoted as G.
[0033] Then, when EC ≤ DC, H I =OI=OH+HI=GC+FC, when EC>DC, H I =OI=HI-OH=FC-GC;
[0034] S11.2, Calculate the silt height H yn =RH I The silt height ratio is ;
[0035] S11.3, Draw the silt surface line in the 3D panoramic image of the pipeline based on the silt height.
[0036] As a preferred option, the number of laser emitters is 5 groups. In step S9, the calculation process of the coordinates of each laser point on the cross-section of the 3D panoramic model of the pipeline is as follows:
[0037] S9.1, the tilt angles of the lasers emitted by the 5 laser emitters are denoted as θ1, θ2, θ3, θ4, and θ5, respectively;
[0038] S9.2, establish a rectangular coordinate system with the center O as the origin, and denote the coordinates of point C as (x, y, y). c ,y c ), , ;
[0039] S9.3, let β be the angle of inclination of the line passing through the laser point and point C, and let β be the intermediate variable. , , , , The coordinates of the two points where the line intersects the circle: , Substitute the tilt angles θ1, θ2, θ3, θ4, and θ5 of each laser into β in the above formula, and select the correct intersection coordinates to obtain the coordinate values of each laser point.
[0040] As a preferred embodiment, the process of generating the 3D panoramic image of the pipeline in step S10 is as follows:
[0041] S10.1. Determine the coordinate system and starting coordinates: Based on the coordinates of each laser point on the cross-section of the three-dimensional panoramic model of the pipeline, calculate the actual coordinates of each laser point in three-dimensional space, thereby determining the starting coordinate position of image E in the three-dimensional panoramic model of the pipeline;
[0042] S10.2. Segmentation and Distance Calculation: Divide each image E obtained in step S6 into n equal parts along the height direction representing the circumferential direction. For each image part, calculate the coordinates of two points on the circumference of the pipe, and then calculate the arc length L between these two points. Qk ;
[0043] S10.3, Adjust image size: For each image segment obtained in step S10.2, adjust its height according to the corresponding arc length L. Qk Adjustments are made while keeping the width constant, thereby generating a series of new images F. ij , where i represents the position index along the circumferential direction of the pipe and j represents the position index along the axial direction of the pipe;
[0044] S10.4. Integrate into the 3D model: Integrate all images F obtained in step S10.3 into the 3D model. ij Place them according to their corresponding positions in the 3D panoramic model of the pipeline, ensuring that each image F ij The edges of the image blend seamlessly with those of other adjacent images, forming a complete and continuous 3D panoramic image of the pipeline.
[0045] An imaging device for implementing the above-mentioned pipeline image processing method includes an imaging device body with multiple mounting positions. A laser emitter and a camera are detachably mounted in their respective mounting positions. The number and arrangement of the laser emitter and camera are selected according to the structural parameters of the pipeline to be inspected and the environmental conditions.
[0046] In summary, the present invention has the following advantages:
[0047] (1) By arranging multiple sets of laser emitters and multiple cameras on the imaging device and using laser points as a reference for rotation and scaling correction, the present invention can ensure accurate alignment of the stitching boundary and obtain a continuous and seamless two-dimensional panoramic image, which can significantly reduce the risk of missed detection and false detection.
[0048] (2) The present invention first turns on the laser emitter to capture an image with laser points, and uses the laser points as a reference to obtain image transformation parameters. Then the laser emitter is turned off, and the original image without laser is captured again as the final stitching material. This can effectively avoid the interference of laser on image capture, so as to obtain a clearer image and more accurate subsequent defect identification.
[0049] (3) In view of the problem of defects and breaks in the first and last images, the present invention divides and rearranges the images so that the defects in the stitched two-dimensional panoramic image are complete and easy to identify and observe.
[0050] (4) This invention uses four cross-shaped ranging modules to calculate the actual radius of the pipeline, realize the accurate modeling of the three-dimensional model of the pipeline, and determine the presence of silt and the silt height ratio through geometric calculation. The silt surface line is visualized and marked in the three-dimensional panoramic image, which can provide a basis for subsequent dredging decisions.
[0051] (5) By segmenting and stretching the image along the circumference, the two-dimensional pixels are accurately mapped to the corresponding curved surface area, overcoming the circumferential compression distortion caused by traditional planar unfolding, and can more realistically present defects in the three-dimensional model. Attached Figure Description
[0052] Figure 1 This is a flowchart of a pipeline image processing method.
[0053] Figure 2 This is a schematic diagram of step S4.
[0054] Figure 3 This is a schematic diagram of a 3D panoramic model of the pipeline.
[0055] Figure 4 This is a schematic diagram illustrating the working principle of the imaging device inside the pipeline.
[0056] Figure 5 This is a schematic diagram of the geometric relationship of the pipe cross section corresponding to the distance measured by the ranging module under a certain working state.
[0057] Figure 6 This is a schematic diagram of the geometric relationship of the pipe cross section corresponding to the distance measured by the ranging module under another working state.
[0058] Figure 7 This is a schematic diagram of image E segmentation in step S10.2.
[0059] Figure 8 This is a schematic diagram comparing the last image E with the first image E to show whether there are defects.
[0060] Figure 9 This is a schematic diagram of image G segmentation in step S7.2.
[0061] Figure 10 This is a schematic diagram of image H obtained in step S7.2.
[0062] Figure 11 This is a schematic diagram of image G segmentation in step S7.3.
[0063] Figure 12This is a schematic diagram of image I obtained in step S7.3.
[0064] Figure 13 This is a schematic diagram of the front end of the imaging device.
[0065] Figure 14 This is a side view of the imaging device.
[0066] Figure 15 This is a schematic diagram showing the arrangement of each ranging module and angle sensor.
[0067] Figure 16 This is a hardware connection diagram for the imaging device.
[0068] Among them, 1 is a camera, 2 is a laser emitter, 3 is a ranging module, and 4 is an angle sensor. Detailed Implementation
[0069] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.
[0070] Example 1
[0071] like Figure 1 As shown, a pipeline image processing method includes the following steps:
[0072] S1. Multiple laser emitters and multiple cameras are set on the imaging device. Multiple laser points are emitted along the circumferential direction of the inner wall of the pipe by the laser emitters. Each set of laser points includes two laser points. Multiple images are captured along the circumferential direction of the inner wall of the pipe by the cameras, and it is ensured that there is a set of the same laser points in every two adjacent images. Thus, there are two sets of laser points in each image.
[0073] S2, each original image captured is recorded as image A. Each image A is rotated and scaled so that the line segment formed by the same set of two laser points is rotated to the horizontal direction and the line segment length is consistent in the transformed image, thus obtaining image B. The rotation angle value, scaling ratio value and coordinate values of the four laser points in image B are saved for each image B.
[0074] S3, turn off the laser emitter, and the camera takes another image of the inner wall of the pipe, which is recorded as image C;
[0075] S4. Based on the rotation angle value and scaling ratio value obtained in step S2, perform the same rotation and scaling transformation on the corresponding image C to obtain image D;
[0076] S5. Based on the coordinates of the four laser points obtained in step S2, locate the corresponding four points in image D, use these four points as vertices to extract a quadrilateral region, and obtain image E.
[0077] S6. Stitch the images E sequentially to obtain a two-dimensional pipe stitched panoramic image, denoted as image G.
[0078] In this embodiment, there are 5 cameras, and the captured images are designated as image 1, image 2, image 3, image 4, and image 5. There are 5 groups of laser points, designated as laser point 1, laser point 2, laser point 3, laser point 4, and laser point 5. For example... Figure 2 As shown, the heights of the transformed images E obtained from images 1 and 5 are H1 and H2, respectively, and the width of both is W.
[0079] Example 2
[0080] In conjunction with Example 1, a target detection algorithm is used to identify defects in image G, thereby obtaining the number and size of defects.
[0081] Specifically, if there are no defects in the panoramic image of the pipeline that span the last image E and the first image E, such as Figure 8 As shown in (a) above, the defect identification step ends;
[0082] If there is a defect that spans between the last image E and the first image E, such as Figure 8 As shown in (b) above, the following steps are performed:
[0083] S7.1, with the length representing the circumferential direction of the pipe as the ordinate and the axial direction of the pipe as the abscissa, find a horizontal line with the ordinate y1 in the image G, requiring that no defects cross this horizontal line.
[0084] The method for finding the horizontal line with ordinate y1 is as follows: The target recognition algorithm obtains the top-left corner coordinates (xi, yi), defect width Wi, and height Hi of all defects, where i represents the i-th defect. Let the variable YY loop starting from 0, incrementing by 1 after each loop until the height of image G is reached. The loop content is as follows: Draw a horizontal auxiliary line using variable YY as the ordinate; determine whether this horizontal auxiliary line passes through the i-th defect, with the condition YY≥yi and YY≤yi+Hi; if this horizontal auxiliary line does not pass through all defects, then variable YY is the ordinate y1.
[0085] If a horizontal line that meets the requirements exists in image G, then proceed to step S7.2; otherwise, proceed to steps S7.3 to S7.6.
[0086] S7.2, as Figure 9 As shown, the image G is divided into two images, G1 and G2, by a horizontal line with the vertical coordinate y1. These images are then stitched together in the order of G2 and G1 to form a single image H, with the lower edge of G2 coinciding with the upper edge of G1. Figure 10 As shown, image H is obtained;
[0087] S7.3, such as Figure 11 As shown, the target detection algorithm identifies defects in image G from top to bottom. The lowest ordinate of the first complete defect (defect 1) is y2. The horizontal line with the ordinate y2 divides image G into two images, G3 and G4.
[0088] S7.4 In image G, starting from the second complete defect, find a defect (defect 2) that passes through the horizontal line y2 and has the highest top. The top of this defect is marked with the vertical coordinate y3. Divide image G3 into two images G31 and G32 by the horizontal line with the vertical coordinate y3.
[0089] S7.5, such as Figure 12 As shown, images I are stitched together in the top-to-bottom order of G32, G4, and G3;
[0090] S7.6 Mark the repeated incomplete defects in area G32 of image I in red, and mark the remaining defects in green.
[0091] In the stitched image I, all defects have complete shapes, such as defect 1, defect 2, and defect 3, while in G32, duplicate incomplete defects appear, which are marked in red. Figure 12 The portion selected in the middle frame will not be considered in subsequent defect assessments. Other defects are marked in green, indicating that all defects are in their complete form.
[0092] Specifically, the method for determining whether a defect spans the last image and the first image is as follows: the last image E and the first image E are stitched together to form an image F. The object detection algorithm is used to identify the location and size values of the defect. Combined with the dimensions of the two images, it is determined whether a defect spans the two images.
[0093] Example 3
[0094] In conjunction with Embodiment 1, the pipeline image processing method further includes the following steps:
[0095] S8, such as Figure 3 As shown, an angle sensor and four ranging modules are set on the imaging device. The ranging directions of the four ranging modules are distributed in a cross shape within the same pipe cross section. The pipe radius R is calculated based on the distance values measured by the ranging modules. Then, based on the radius R and the width W of the image E, a three-dimensional panoramic model of the pipe with radius R and axial length W is constructed.
[0096] S9, based on the tilt angle of the laser emitter, calculate the coordinates of each laser point on the cross-section of the three-dimensional panoramic model of the pipeline to determine the start and end positions of each image E in the circumferential direction of the pipeline.
[0097] S10, geometrically correct image E according to the circumference arc length of the pipeline, and map the corrected image to the corresponding curved surface position of the 3D panoramic model to generate a 3D panoramic image of the pipeline.
[0098] In use, the imaging device is mounted on the pipeline inspection robot, and its posture is adjustable. After the pipeline inspection robot reaches the designated location on the pipeline, before taking a picture, the controller reads the tilt angle of the pipeline imaging device from the angle sensor and adjusts the device to a horizontal state, that is, the first ranging module and the third ranging module are on the same horizontal line, before taking a picture.
[0099] Example 4
[0100] In conjunction with Example 3, when the pipe to be inspected is a horizontal or inclined pipe, silt deposits may be present at the bottom of the pipe, such as... Figures 5-6 As shown, the pipe radius is calculated using data obtained from the ranging module, and the presence of silt at the bottom of the pipe is determined. The specific process is as follows:
[0101] S8.1, the four ranging modules are arranged circumferentially as the first ranging module, the second ranging module, the third ranging module, and the fourth ranging module. The measured distances are denoted as L1, L2, L3, and L4, where the second ranging module measures directly downwards. The center point around which the four ranging modules are arranged is denoted as C, and the distance from the four ranging modules to the center point is denoted as r. The ranging points of the first, second, third, and fourth ranging modules inside the pipe are denoted as M, F, N, and D, respectively. Then, MC = L1 + r, FC = L2 + r, NC = L3 + r, and DC = L4 + r. When there is silt at the bottom of the pipe, the measured distance L2 is the distance between the second ranging module and the surface of the silt.
[0102] S8.2, the point on the inner wall of the pipe where the second ranging module extends along the ranging direction is denoted as E. According to the power theorem, ;
[0103] S8.3, obtain the pipe radius R, ;
[0104] When EC is greater than FC, it is determined that there is silt at the bottom of the pipeline. At this time, the measuring point F is the surface of the silt. When EC equals FC, there is no silt at the bottom of the pipeline. At this time, the measuring point F coincides with E.
[0105] Specifically, when there is silt at the bottom of the pipe, the silt height and silt height ratio are calculated. The specific process is as follows:
[0106] S11.1, the line on the surface of the silt at the bottom of the pipe is denoted as line segment AB, and the center point of the pipe is denoted as O; the perpendicular line from O to line segment MN is denoted as H, and the intersection of the extension of line segment OH and line segment AB is denoted as I; the perpendicular line from O to line segment DE is denoted as G.
[0107] Then there are, such as Figure 5 As shown, when EC≤DC, H I =OI=OH+HI=GC+FC, such as Figure 6 As shown, when EC > DC, H I =OI=HI-OH=FC-GC;
[0108] S11.2, Calculate the silt height H yn =RH I The silt height ratio is ;
[0109] S11.3, Draw the silt surface line in the 3D panoramic image of the pipeline based on the silt height.
[0110] Example 5
[0111] Referring to Example 4, the number of laser emitters is 5 groups. In step S9, the calculation process of the coordinates of each laser point on the cross-section of the 3D panoramic model of the pipeline is as follows:
[0112] S9.1, the tilt angles of the lasers emitted by the 5 laser emitters are denoted as θ1, θ2, θ3, θ4, and θ5, respectively;
[0113] S9.2, establish a rectangular coordinate system with the center O as the origin, and denote the coordinates of point C as (x, y, y). c ,y c ), , ;
[0114] S9.3, let β be the angle of inclination of the line passing through the laser point and point C, and let β be the intermediate variable. , , , , The coordinates of the two points where the line intersects the circle: , Substitute the tilt angles θ1, θ2, θ3, θ4, and θ5 of each laser into β in the above formula, and select the correct intersection coordinates to obtain the coordinate values of each laser point.
[0115] In step S9.1 above, the emission angle of each laser emitter was determined during the manufacturing of the imaging device. For a pipe imaging device with 5 cameras, there are also 5 sets of corresponding laser points, with the angle between adjacent laser points being 360° / 5. The 5 laser points arranged circumferentially correspond to points R, S, T, U, and V on the inner wall of the pipe. The laser emitters corresponding to these 5 laser points are denoted as laser emitters 1, 2, 3, 4, and 5. The angle between the laser line segment RC corresponding to laser emitter 1 and the horizontal line is denoted as α, where ∠RCS=∠SCT=∠TCU=∠UCV=∠VCR=360° / 5=72°. Figure 4 As shown. The tilt angles of lines RC, SC, TC, UC, and VC are θ1, θ2, θ3, θ4, and θ5, respectively, where θ1 = α, θ2 = θ1 + 72°, θ3 = θ1 + 72°, θ4 = θ1 + 72°, and θ5 = θ1 + 72°.
[0116] Specifically, in step S10, the process of generating a 3D panoramic image of the pipeline is as follows:
[0117] S10.1. Determine the coordinate system and starting coordinates: Based on the coordinates of each laser point on the cross-section of the three-dimensional panoramic model of the pipeline, calculate the actual coordinates of each laser point in three-dimensional space, thereby determining the starting coordinate position of image E in the three-dimensional panoramic model of the pipeline;
[0118] S10.2. Segmentation and Distance Calculation: Divide each image E obtained in step S6 into n equal parts along the height direction representing the circumferential direction. For each image part, calculate the coordinates of two points on the circumference of the pipe, and then calculate the arc length L between these two points. Qk ;
[0119] like Figure 7 As shown, the RV region corresponds to the effective shooting area of the first camera. R and V correspond to the laser points at both ends. Since the arc RV is not proportionally compressed into the line segment RV, the corresponding image cannot be proportionally stretched back to the arc RV. Here, the image E is first divided into n equal parts along the direction of line segment RV, and then each part is stretched onto the arc RV, which better restores the true proportional image of the arc RV. Taking the division into 5 equal parts as an example, the first image corresponds to the projection of arc RQ1 onto line segment RP1, the second image corresponds to the projection of arc Q1Q2 onto line segment P1P2, the third image corresponds to the projection of arc Q2Q3 onto line segment P2P3, the fourth image corresponds to the projection of arc Q3Q4 onto line segment P3P4, and the fifth image corresponds to the projection of arc Q4V onto line segment P4V.
[0120] The specific process of step S10.2 above is as follows:
[0121] (1) Place two adjacent laser points R(x) R ,yR ) and V(x V ,y V The line segment RV is divided into n equal parts, and the division point P is obtained. k (where k = 1, 2, ..., n-1), its coordinates are (x k ,y k ). , .
[0122] (2) From the center O (0,0) through point P k Draw a ray that intersects the circle at point Q. k Its coordinates are (x Qk ,y Qk ). , .
[0123] (3) Find the distance between two adjacent points. This includes the lengths of line segments RQ1, Q1Q2, Q2Q3, Q3Q4, and Q4V.
[0124] S10.3, Adjust image size: For each image segment obtained in step S10.2, adjust its height according to the corresponding arc length L. Qk Adjustments are made while keeping the width constant, thereby generating a series of new images F. ij Where i represents the position index along the circumference of the pipe, and j represents the position index along the axial direction of the pipe; that is, the height of the first image is scaled to the line segment L. RQ1 The length of the image F is obtained. 11 The height of the second image is scaled to line segment L. Q1Q2 The length of the image F is obtained. 12 The height of the third image is scaled to line segment L. Q2Q3 The length of the image F is obtained. 13 The height of the fourth image is scaled to line segment L. Q3Q4 The length of the image F is obtained. 14 The height of the fifth image is scaled to line segment L. Q4V The length of the image F is obtained. 15 .
[0125] S10.4. Integrate into the 3D model: Integrate all images F obtained in step S10.3 into the 3D model. ij Place them according to their corresponding positions in the 3D panoramic model of the pipeline, ensuring that each image F ij The edges of the image seamlessly blend with adjacent images, forming a complete and continuous 3D panoramic image of the pipeline. That is, image F... 11 Place the image at position RQ1 in the 3D image of the pipeline, and set image F... 12Place the image at position Q1Q2 in the 3D image of the pipeline, and set image F... 13 Place the image at position Q2Q3 in the 3D image of the pipe, and set image F... 14 Place the image at position Q3Q4 in the 3D image of the pipeline, and set image F... 15 Place it at position Q4V in the 3D image of the pipeline. Process other images similarly to obtain a panoramic image of the pipeline.
[0126] Example 6
[0127] like Figures 13-16 As shown, an imaging device is used to implement the pipeline image processing method of Embodiment 1. It includes an imaging device body with multiple mounting positions. A laser emitter and a camera are detachably mounted in the corresponding mounting positions. The number and arrangement of the laser emitter and camera are selected according to the structural parameters of the pipeline to be detected and the environmental conditions.
[0128] The above embodiments are preferred embodiments of the invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.
Claims
1. A pipeline image processing method, characterized in that: Includes the following steps, S1. Multiple laser emitters and multiple cameras are set on the imaging device. Multiple laser points are emitted along the circumferential direction of the inner wall of the pipe by the laser emitters. Each set of laser points includes two laser points. Multiple images are captured along the circumferential direction of the inner wall of the pipe by the cameras, and it is ensured that there is a set of the same laser points in every two adjacent images. Thus, there are two sets of laser points in each image. S2, each original image captured is recorded as image A. Each image A is rotated and scaled so that the line segment formed by the same set of two laser points is rotated to the horizontal direction and the line segment length is consistent in the transformed image, thus obtaining image B. The rotation angle value, scaling ratio value and coordinate values of the four laser points in image B are saved for each image B. S3, turn off the laser emitter, and the camera takes another image of the inner wall of the pipe, which is recorded as image C; S4. Based on the rotation angle value and scaling ratio value obtained in step S2, perform the same rotation and scaling transformation on the corresponding image C to obtain image D; S5. Based on the coordinates of the four laser points obtained in step S2, locate the corresponding four points in image D, use these four points as vertices to extract a quadrilateral region, and obtain image E. S6. Stitch the images E sequentially to obtain a two-dimensional pipe stitched panoramic image, denoted as image G.
2. The pipeline image processing method according to claim 1, characterized in that: A target detection algorithm is used to identify defects in image G, and the number and size of defects are obtained.
3. The pipeline image processing method according to claim 2, characterized in that: If no defect crosses the last image E and the first image E in the panoramic image of the pipeline, the defect identification step ends. If a defect exists that spans between the last image E and the first image E, then perform the following steps: S7.1, with the length representing the circumferential direction of the pipe as the ordinate and the axial direction of the pipe as the abscissa, find a horizontal line with the ordinate y1 in the image G, requiring that no defects cross this horizontal line. If a horizontal line that meets the requirements exists in image G, then proceed to step S7.2; otherwise, proceed to steps S7.3 to S7.
6. S7.2, divide the image G into two images G1 and G2 by the horizontal line with the vertical coordinate y1, stitch G2 and G1 together in order to form an image H, and make the lower edge of G2 connect with the upper edge of G1 to obtain image H; S7.3, the target detection algorithm identifies defects in image G from top to bottom. The lowest ordinate of the first complete defect is y2. The horizontal line with the ordinate y2 divides image G into two images, G3 and G4. S7.4 In image G, starting from the second complete defect, find a defect that passes through the horizontal line y2 and has the highest top. The top of this defect is marked with the vertical coordinate y3. Divide image G3 into two images G31 and G32 by the horizontal line with the vertical coordinate y3. S7.5, stitch together G32, G4, and G3 in the top-to-bottom order to form an image I; S7.6 Mark the repeated incomplete defects in area G32 of image I in red, and mark the remaining defects in green.
4. The pipeline image processing method according to claim 3, characterized in that: The method for determining whether a defect spans the last image and the first image is as follows: the last image E and the first image E are stitched together to form an image F. The location and size values of the defect are identified using an object detection algorithm. Combined with the sizes of the two images, it is determined whether a defect spans the two images.
5. The pipeline image processing method according to claim 1, characterized in that: It also includes the following steps, S8. An angle sensor and four ranging modules are set on the imaging device. The ranging directions of the four ranging modules are distributed in a cross shape within the same pipe cross section. The pipe radius R is calculated based on the distance value measured by the ranging modules. Then, based on the radius R and the width W of the image E, a three-dimensional panoramic model of the pipe with radius R and axial length W is constructed. S9, based on the tilt angle of the laser emitter, calculate the coordinates of each laser point on the cross-section of the three-dimensional panoramic model of the pipeline to determine the start and end positions of each image E in the circumferential direction of the pipeline. S10, geometrically correct image E according to the circumference arc length of the pipeline, and map the corrected image to the corresponding curved surface position of the 3D panoramic model to generate a 3D panoramic image of the pipeline.
6. The pipeline image processing method according to claim 5, characterized in that: The pipe radius is calculated using data obtained from the ranging module, and the presence of silt at the bottom of the pipe is determined. The specific process is as follows. S8.1, the four ranging modules are arranged in the circumferential direction as the first ranging module, the second ranging module, the third ranging module, and the fourth ranging module. The measured distances are denoted as L1, L2, L3, and L4. The second ranging module measures distances directly downwards. The center point around which the four ranging modules are arranged is denoted as C, and the distance from the four ranging modules to the center point is denoted as r. The ranging points of the first, second, third, and fourth ranging modules inside the pipe are denoted as M, F, N, and D, respectively. Then, we have MC = L1 + r, FC = L2 + r, NC = L3 + r, and DC = L4 + r. S8.2, the point on the inner wall of the pipe where the second ranging module extends along the ranging direction is denoted as E. According to the power theorem, ; S8.3, obtain the pipe radius R, ; When EC is greater than FC, it is determined that there is silt at the bottom of the pipeline. At this time, the measuring point F is the surface of the silt. When EC equals FC, there is no silt at the bottom of the pipeline. At this time, the measuring point F coincides with E.
7. The pipeline image processing method according to claim 6, characterized in that: When there is silt at the bottom of the pipe, calculate the silt height and silt height ratio. The specific process is as follows: S11.1, the line on the surface of the silt at the bottom of the pipe is denoted as line segment AB, and the center point of the pipe is denoted as O; the perpendicular line from O to line segment MN is denoted as H, and the intersection of the extension of line segment OH and line segment AB is denoted as I; the perpendicular line from O to line segment DE is denoted as G. then, when EC≤DC, H I = OI = OH + HI = GC + FC, when EC>DC, H I = OI = HI - OH = FC - GC; S11.2, Calculate the silt height H yn =RH I The silt height ratio is ; S11.3, Draw the silt surface line in the 3D panoramic image of the pipeline based on the silt height.
8. The pipeline image processing method according to claim 6, characterized in that: There are 5 sets of laser emitters. In step S9, the calculation process of the coordinates of each laser point on the cross-section of the 3D panoramic model of the pipeline is as follows. S9.1, the tilt angles of the lasers emitted by the 5 laser emitters are denoted as θ1, θ2, θ3, θ4, and θ5, respectively; S9.2, establish a rectangular coordinate system with the center O as the origin, and denote the coordinates of point C as (x, y, y). c ,y c ), , ; S9.3, let β be the angle of inclination of the line passing through the laser point and point C, and let β be the intermediate variable. , , , , The coordinates of the two points where the line intersects the circle: , Substitute the tilt angles θ1, θ2, θ3, θ4, and θ5 of each laser into β in the above formula, and select the correct intersection coordinates to obtain the coordinate values of each laser point.
9. The pipeline image processing method according to claim 1, characterized in that: In step S10, the process of generating a 3D panoramic image of the pipeline is as follows: S10.
1. Determine the coordinate system and starting coordinates: Based on the coordinates of each laser point on the cross-section of the three-dimensional panoramic model of the pipeline, calculate the actual coordinates of each laser point in three-dimensional space, thereby determining the starting coordinate position of image E in the three-dimensional panoramic model of the pipeline; S10.
2. Segmentation and distance calculation: each image E obtained in step S6 is segmented into n parts along the height direction representing the circumferential direction, and for each part, the coordinates of two points on the pipe circumference are calculated, and the arc length L between the two points is calculated accordingly Qk ; S10.3, Adjust image size: For each image segment obtained in step S10.2, adjust its height according to the corresponding arc length L. Qk Adjustments are made while keeping the width constant, thereby generating a series of new images F. ij , where i represents the position index along the circumferential direction of the pipe and j represents the position index along the axial direction of the pipe; S10.
4. Integrate into the 3D model: Integrate all images F obtained in step S10.3 into the 3D model. ij Place them according to their corresponding positions in the 3D panoramic model of the pipeline, ensuring that each image F ij The edges of the image blend seamlessly with those of other adjacent images, forming a complete and continuous 3D panoramic image of the pipeline.
10. An imaging device for implementing the pipeline image processing method according to any one of claims 1 to 9, characterized in that: The system includes an imaging device body with multiple mounting positions. Laser emitters and cameras are detachably mounted in their respective mounting positions. The number and arrangement of laser emitters and cameras are selected based on the structural parameters of the pipeline to be inspected and the environmental conditions.