Apparatus for detecting and imaging three-dimensional structures inside a pipe and information processing method

Abstract

The present application belongs to the field of optical detection and optical imaging technology, and relates to pipeline internal nondestructive detection and structure reconstruction, in particular to a device for pipeline internal three-dimensional structure detection and imaging and an information processing method, which overcomes the problem in the prior art that only two-dimensional unfolded images of the pipeline interior can be obtained by using image fusion and splicing methods, and can realize three-dimensional structure measurement and three-dimensional image reconstruction of the pipeline interior. The device is composed of a light source, a multi-camera combination, a mechanical connecting rod, a translation stage, a computer and a stereo chessboard target. The detection process includes camera calibration, random point entity image projection image acquisition and pipeline internal illumination image acquisition. The information processing process includes pipeline internal three-dimensional structure reconstruction and pipeline internal three-dimensional image reconstruction. The present application has the advantages of simple device, convenient operation, high calculation precision and efficiency.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of optical detection and optical imaging, and relates to pipeline internal nondestructive detection and structure reconstruction, in particular to a device for pipeline internal three-dimensional structure detection and imaging and an information processing method. BACKGROUND

[0002] The pipeline is widely used in the fields of energy, power generation, aviation, refrigeration, chemical industry, etc. In the production and manufacturing process, due to the problems of self quality and external force interference, the pipeline may have various defects such as cracks, flaws, deformation, etc. At the same time, due to the influence of various environmental factors such as medium corrosion and mechanical vibration, the pipeline will age, which greatly increases the probability of problems such as cracks, flaking and perforation. Therefore, it is necessary to conduct regular safety monitoring on the pipeline.

[0003] Pipeline detection can be divided into two types: external detection and internal detection. The external detection of the pipeline refers to using relevant detection devices to detect on the outside of the pipeline. This detection method analyzes the relevant data such as the damage of the pipeline anticorrosion layer and human damage, and then evaluates the pipeline loss. However, the corrosion condition of the pipeline itself is difficult to determine by the external detection method. The internal detection method of the pipeline mainly uses relevant detection devices to move inside the pipeline, collects data inside the pipeline and conducts in-depth analysis, which can accurately detect the internal corrosion, physical deformation, cracks and other defect problems of the pipeline. Therefore, the internal detection method of the pipeline has become a highly respected nondestructive testing method in the world, and has become the development direction of the oil and gas pipeline detection industry.

[0004] The image detection method is a simple and efficient internal detection method of the pipeline, which uses an optical imaging device to shoot high-definition images inside the pipeline, and can more intuitively observe the damage information such as flaking and cracking. Due to the limited viewing angle, it is necessary to combine digital image processing technology to fuse and splice multiple images to restore the complete internal image of the pipeline. However, the existing internal detection device of the pipeline uses a forward imaging endoscope to collect images, and combines a post-image splicing algorithm, which can only restore the two-dimensional unfolded image inside the pipeline, and cannot reflect the three-dimensional structure of the pipeline, so it cannot perform quantitative damage positioning test. SUMMARY

[0005] The purpose of the present application is to provide a device for pipeline internal three-dimensional structure detection and imaging and an information processing method, which overcomes the problem that the prior art cannot measure the three-dimensional structure of the pipeline and reconstruct the three-dimensional image structure, so it cannot perform quantitative damage positioning detection.

[0006] To achieve the above purpose, the technical scheme adopted by the present application is as follows:

[0007] The utility model provides a device for pipeline internal three -dimensional structure detection and imaging, which is composed of a pair of pattern projection lamps, a multi-camera combination, a mechanical connecting rod, a translation stage, a computer and a stereo chessboard target, one end of the mechanical connecting rod is connected with the translation stage, the front of the other end is provided with a pattern projection lamp 1, a multi-camera combination and a pattern projection lamp 2 along the axial direction of the axis, and the computer is connected with the pair of pattern projection lamps, the multi-camera combination and the translation stage respectively.

[0008] The stereo chessboard target is a hollow four -sided column with a square cross section, two opposite square surfaces are open, and the four inner surfaces are provided with chessboard targets.

[0009] An information processing method for a device for pipeline internal three -dimensional structure detection and imaging, comprising the following steps:

[0010] Step one, system calibration, multi-angle projection image and illumination image acquisition:

[0011] The system calibration is multi-camera combination calibration using a stereo chessboard target, specifically moving the multi-camera combination to the center position of the stereo chessboard target, taking the position as the initial position of the detection device; the multi-camera combination acquires the image of the stereo chessboard target, and the focal length, rotation matrix and translation vector of each camera are determined by Zhang Zhengyou calibration method as the initial parameters of the detection device;

[0012] Step two, three -dimensional structure reconstruction in pipeline is carried out by using multi-angle projection image;

[0013] Step three, on the basis of the three -dimensional structure reconstructed in step two, three -dimensional image reconstruction of the pipeline inner wall is carried out by using multi-angle illumination image, comprising the following steps:

[0014] S1: the pipeline inner wall of known three -dimensional structure is evenly divided into M discrete grids, and it is assumed that the object point f (x w ,y w ,z w ) in each grid is a constant.

[0015] S2: a matrix IMG (M, N) is generated for storing the reconstructed image, wherein N is the number of cameras;

[0016] S3: for the jth discrete grid, its world coordinate system (x w ,y w ,z w ) can be determined, and the relationship

[0017]

[0018] And

[0019]

[0020] get its image coordinate system position (x' i ,y' i ) in the i-th camera, where the serial number of the camera i = 1, 2, …, N, f i is the focal length of the camera, R i and T i are the rotation matrix and translation vector of the camera.

[0021] S4: If the image coordinate system (x' i ,y' i ) is in the imaging range of the camera, then the pipe wall image of the i-th camera is processed, and the value of IMG(j, i) is determined by the method of bilinear interpolation; otherwise, IMG(j, i) is assigned a value of 0.

[0022] S5: Repeat S3 and S4 until the matrix IMG(M, N) is completely assigned, and N pipe wall images are generated;

[0023] S6: Perform image fusion processing on the N pipe wall images to obtain the final pipe internal three-dimensional image result.

[0024] Further, the detection process of step one includes the following steps:

[0025] S1: Use a stereo chessboard target for multi-camera combined calibration;

[0026] S2: Replace the stereo chessboard target with the measured pipe, and the center of the pipe coincides with the axis of the connecting rod.

[0027] S3: Use a computer to control the pattern projection lamp to project a random point entity image on the pipe wall, and use a multi-camera combination to simultaneously collect projection images of different angles inside the pipe; then control the pattern projection lamp to uniformly illuminate the pipe wall, and use a multi-camera combination to collect the illumination image of the pipe wall.

[0028] S4: Control the pattern projection lamp and the multi-camera combination to move inside the pipe, and repeat S3 at different depth positions until the pipe is completely detected.

[0029] Further, the specific steps of step two are: the process of using the projection image to reconstruct the three-dimensional structure inside the pipe includes the following steps:

[0030] S1: Perform feature point detection and matching on the projection random point entity images collected by the adjacent two cameras, and obtain the image coordinate system positions of the matching projection feature point pairs in the two projection images;

[0031] S2: Preprocess the projection feature point pairs and remove the obviously mismatched feature point pairs;

[0032] S3: any one of the remaining feature points, which are imaged in the i-th and i+1-th cameras, the image coordinate systems in the two projection images are (x' i ,y' i ) and (x' i+1 ,y' i+1 ) respectively, the focal lengths of the two cameras are f i and f i+1 , the rotation matrix R and the translation vector T are

[0033]

[0034]

[0035] The world coordinate system (x w ,y w ,z w ) and the image coordinate systems (x' i ,y' i ), (x' i+1 ,y' i+1 ) of the feature points satisfy the following relationships:

[0036] The above equations are established by using the image coordinate systems of the feature points and the camera parameters, and the world coordinate system position (x w ,y w ,z w ) of the feature point can be obtained by solving the equations;

[0037] S4: the world coordinate system positions of all feature points are obtained by S3, and then the surface fitting is performed to obtain the three-dimensional structure of the pipeline.

[0038] Compared with the prior art, the present application has the following advantages and effects:

[0039] 1. In view of the problem of insufficient feature point information on the inner wall of the pipeline, the pattern projection lamp is applied in the device, which can illuminate and project random point entities on the inner wall of the pipeline, the projection image completely covers the inner wall of the pipeline, and the complete reconstruction of the three-dimensional structure of the pipeline is realized by completely covering the feature points in the pipeline.

[0040] 2. The fixed position multi-camera combination is applied in the method, which can realize the simultaneous acquisition of multiple angle images, improve the detection efficiency, and the spatial positions of the multiple cameras are strictly fixed, so that the position deviation caused by the movement in the acquisition process is avoided, and the cuboid three-dimensional chessboard target structure is proposed, which can realize the calibration of the 360° distributed multi-camera combination, so that the detection precision can be greatly improved.

[0041] 3. The method of the present application can reconstruct the three-dimensional coordinates of the feature points in the adjacent cameras using the pixel positions of the feature points in the projected images, and can fit the three-dimensional structure of the pipeline using the three-dimensional coordinates of numerous feature points. That is, the present application first detects the three-dimensional structure inside the pipeline, and then reconstructs the three-dimensional image of the inner wall based on the three-dimensional structure, so that the reconstructed image contains three-dimensional position information, which is more conducive to the quantitative positioning of the pipeline inner wall damage, and is of great significance for the practical application of the pipeline internal detection technology.

[0042] 4. The measurement method is simple to operate: when the connecting rod is placed in the measured pipeline, it only needs to ensure that the center of the pipeline is basically coincident with the axis of the connecting rod from the human senses, and the present application does not need to be strictly coincident, thereby reducing the difficulty of operation and improving the efficiency of measurement. BRIEF DESCRIPTION OF DRAWINGS

[0043] Figure 1 is a structural diagram of a pipeline internal three-dimensional structure detection and imaging device;

[0044] Figure 2 is a structural diagram of a camera calibration target;

[0045] Figure 3 is an example of a projection random point entity diagram;

[0046] Figure 4 is a coordinate system diagram for pipeline three-dimensional structure reconstruction;

[0047] Figure 5 is a pipeline three-dimensional structure reconstruction flowchart;

[0048] Figure 6 is a pipeline inner wall three-dimensional image reconstruction flowchart. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solutions and advantages of the present application clearer and more apparent, the present application will be further described in detail below in combination with the drawings and examples. It should be understood that the specific examples described herein are only used to explain the present application and do not limit the present application.

[0050] Example: The present application provides a device for pipeline internal three-dimensional structure detection and imaging, as shown in Figure 1 The device is composed of a pair of pattern projection lamps, a multi-camera combination, a mechanical connecting rod, a translation stage, a computer and a stereo chessboard target. One end of the mechanical connecting rod is connected to the translation stage, and the front part of the other end is provided with a pattern projection lamp 1, a multi-camera combination and a pattern projection lamp 2 along the axial direction. The computer is connected with the pair of pattern projection lamps, the multi-camera combination and the translation stage, respectively.

[0051] The stereo chessboard target structure used for camera calibration is as shown in Figure 2The hollow four-sided cylinder is square in cross-section, with two opposite square faces open for the pattern projection lamp and multi-camera combination to enter the interior of the stereo target, and the other four inner surfaces are made of the checkerboard pattern commonly used in camera calibration technology, i.e. the checkerboard target. The center of the hollow four-sided cylinder is defined as the origin O of the world coordinate system (x, y, z), and the three axes of the world coordinate system are parallel to the three edges of the cuboid. The four checkerboard target planes are x = D, x = -D, y = D and y = -D, respectively, for calibrating cameras at different positions. w w w w w w w w

[0052] The functions of the parts are as follows:

[0053] (1) Pattern projection lamp: used for illumination inside the pipeline and projection of random point entities on the inner wall of the pipeline. In order to ensure uniformity of illumination brightness, pattern projection lamp 1 and pattern projection lamp 2 are respectively installed before and after the motion axis of the multi-camera combination composed of multiple cameras.

[0054] (2) Multi-camera combination: used for image acquisition inside the pipeline. Multiple cameras are uniformly distributed at equal angles on a circular ring in the same plane, and the imaging planes of the cameras are placed vertically outward along the radius of the circular ring. In order to ensure a sufficient number of matching feature points between the images of adjacent cameras, the images of adjacent two cameras should have a large enough intersection area, and at the same time, in order to comprehensively balance the detection efficiency, the images acquired by adjacent two cameras should have a half area overlap. The number of cameras can be adjusted according to the size of the field of view angle of a single camera.

[0055] (3) Mechanical connecting rod: the pattern projection lamp and the multi-camera combination are connected and fixed on the translation stage through the mechanical connecting rod.

[0056] (4) Translation stage: the motion direction of the translation stage is parallel to the direction of the pipeline, and the images of different depths inside the pipeline are acquired by controlling the motion of the camera combination through the translation stage.

[0057] (5) Computer: used for driving and control of the light source, multi-camera combination and translation stage, image acquisition and storage, information processing and detection result display.

[0058] (6) Stereo checkerboard target: the position and parameters of the checkerboard target on each face can be accurately determined, and the two opposite openings of the four-sided cylinder are used for the pattern projection lamp and the multi-camera combination to enter the interior of the target structure, for camera parameter calibration of the 360° distributed multi-camera combination.

[0059] ​​​​​​​​The process of using the above-mentioned device to perform three-dimensional structural inspection and imaging of the inside of a pipeline includes the following steps:

[0060] Step 1: System calibration, acquisition of multi-angle projection images and illumination images:

[0061] S1: Perform multi-camera setup calibration using a 3D checkerboard target. Move the multi-camera setup to the center of the 3D checkerboard target, which is the center of the hollow tetrahedron. In the world coordinate system (x... w ,y w ,z w The origin O w The position is used as the initial position of the detection device; multiple cameras are combined to acquire images of the three-dimensional chessboard target, and the focal length, rotation matrix and translation vector of each camera are determined using the Zhang Zhengyou calibration method, which are used as the initial parameters of the detection device.

[0062] During camera calibration, the plane containing the multi-camera combination is adjusted to the z-axis using a translation stage. w The initial detection position is on the plane where 0 = 0. Assume the system has N cameras, acquiring target images corresponding to each camera, and determining the focal length f of each camera using Zhang Zhengyou's calibration method. i Rotation matrix R i Translation vector T i , where i = 1, 2, ..., N.

[0063] S2: Replace the 3D chessboard target with the pipe being tested, with the center of the pipe basically coinciding with the axis of the connecting rod.

[0064] S3: Using a computer-controlled pattern projection lamp to project a random point entity image onto the inner wall of the pipe. An example diagram could be shown below. Figure 3 As shown, multiple cameras are used to simultaneously capture projected images of the pipe's interior from different angles. Then, the pattern projection lamp is switched to a uniform illumination mode to illuminate the pipe's inner wall, and multiple cameras are used to capture the illuminated images of the pipe's inner wall.

[0065] S4: Control a pair of pattern projection lights and a combination of multiple cameras to move inside the pipe using a translation stage, repeating S3 at different depth positions until the pipe is completely inspected.

[0066] Step 2: Reconstruct the 3D structure inside the pipe using multi-angle projection images:

[0067] like Figure 4 As shown, assume a point (x) in the world coordinate system w ,y w ,z w Imaging in the i-th and (i+1)-th cameras, since the rotation matrix R of these two cameras has been determined through camera calibration. i ,Ri+1 and translation vector T i i+1 Then the position of the point in the two camera coordinate systems can be determined, respectively as:

[0068]

[0069]

[0070] The focal lengths of the two cameras are f i and f i+1 According to the pinhole camera imaging model, the imaging points of the point in the two cameras are respectively:

[0071]

[0072]

[0073] According to the above relationship, the imaging point positions of the point in the two cameras are:

[0074]

[0075] If the imaging positions of a pair of feature points in the two cameras are measured, the above equation set can be established in combination with the camera parameters, and the world coordinate system position of the feature point can be obtained by solving the equation set.

[0076] According to the above theory, the process of using the projection images to reconstruct the three-dimensional structure inside the pipeline is as shown in Figure 5 , mainly including the following steps:

[0077] S1: For the projection images collected by the adjacent two cameras, the SIFT (Scale Invariant Feature Transform) algorithm is used for feature point detection and matching to obtain the image coordinate system positions of the matched projection feature point pairs in the two images;

[0078] S2: The RANSAC (Random Sample Consensus) algorithm is used to pre-process the projection feature point pairs to eliminate the obviously mismatched feature point pairs;

[0079] S3: For any one of the remaining feature points, the imaging of the feature point in the i th and i+1 th cameras is (x' i , y' i ) and (x' i+1 , y' i+1 ), and the focal lengths of the two cameras are f i and f​i+1 , the rotation matrix R and the T translation vector are respectively

[0080]

[0081]

[0082] According to the pinhole camera imaging model, the world coordinate system (x w ,y w ,z w ) and the image coordinate system (x' i ,y' i ), (x' i+1 ,y' i+1 ) of the feature point satisfy the relationship:

[0083]

[0084] Solving the above equation set can obtain the world coordinate system position (x w ,y w ,z w ) of the feature point.

[0085] S4: using S3 to obtain the world coordinate system position of all feature points, and then performing surface fitting, the three-dimensional structure of the pipeline inside can be obtained.

[0086] Step three, on the basis of the three-dimensional structure reconstructed in step two, the multi-angle illumination image is used to perform three-dimensional image reconstruction of the pipeline inner wall:

[0087] The illumination image of the pipeline inside under the condition of uniform illumination collected by the multi-camera combination is used to perform three-dimensional image reconstruction of the pipeline inside, and the flow chart is as shown in Figure 6 , mainly including the following steps:

[0088] S1: the pipeline inner wall with known three-dimensional structure is evenly divided into M discrete grids, and it is assumed that the object point f(x w ,y w ,z w ) in each grid is a constant.

[0089] S2: a matrix IMG(M, N) is generated for storing the reconstructed image, wherein N is the number of cameras;

[0090] S3: for the jth discrete grid, its world coordinate system (x w ,y w ,z w ) can be determined, and the relationship

[0091]

[0092] and

[0093]

[0094] The image coordinate system position (x' i ,y' i ) in the i-th camera can be obtained, wherein the serial number of the camera is i = 1, 2, …, N, f i is the focal length of the camera, R i and T i are the rotation matrix and translation vector of the camera.

[0095] S4: If the image coordinate system (x' i ,y' i ) is in the imaging range of the camera, the pipe wall image of the i-th camera is processed, and the value of IMG(j, i) is determined by the method of bilinear interpolation; otherwise, IMG(j, i) is assigned a value of 0.

[0096] S5: Repeat S3 and S4 until the matrix IMG(M, N) is completely assigned, and N pipe wall images are generated.

[0097] S6: The N pipe wall images are processed by image fusion, and the final three-dimensional damage image of the pipe interior can be obtained.

[0098] The above is only a preferred embodiment of the present application, and is not used to limit the protection scope of the present application. Any equivalent structural changes made according to the content of the specification and drawings of the present application should be included in the patent protection scope of the present application.

Claims

1. An information processing method of an apparatus for internal three-dimensional structure detection and imaging of a pipe, characterized by, The method comprises the following steps: Step one, system calibration, multi-angle projection image and illumination image acquisition: The system calibration is a multi-camera combined calibration using a stereo chessboard target, specifically, the multi-camera combination is moved to the center position of the stereo chessboard target, and the position is taken as the initial position of the detection device; the multi-camera combination acquires the image of the stereo chessboard target, and the focal length, rotation matrix and translation vector of each camera are determined by using Zhang Zhengyou calibration method, serving as the initial parameters of the detection device; Step two, three-dimensional structure reconstruction of the pipeline interior using the multi-angle projection image; Step three, three-dimensional image reconstruction of the pipeline inner wall using the multi-angle illumination image based on the three-dimensional structure reconstructed in step two, comprising the following steps: S1: The inner wall of the pipe with known three-dimensional structure is evenly divided into M discrete grids, and it is assumed that the material point f(x w ,y w ,z w ) in each grid is a constant; S2: generating a matrix IMG(M, N) for storing the reconstructed image, wherein N is the number of cameras; S3: For the jth discrete grid, its world coordinate system (x w ,y w ,z w ) can be determined using the relationship S4: The world coordinate system of the jth discrete grid is determined by the relationship S5: The world coordinate system of the jth discrete grid is determined by the relationship S6: The And its image coordinate system position (x i ,y i ) in the i-th camera, where the camera's index i = 1, 2,..., N, f i is the camera's focal length, R i and T i are the camera's rotation matrix and translation vector; S4: if the image coordinate system (x' i ,y' i ) is in the imaging range of the camera, then the pipe inner wall image of the i-th camera is processed, and the value of IMG(j, i) is determined by the method of bilinear interpolation; otherwise, IMG(j, i) is assigned a value of 0; S5: repeating S3 and S4 until the matrix IMG(M, N) is completely assigned, generating N pipeline inner wall images; S6: performing image fusion processing on the N pipeline inner wall images to obtain the final pipeline interior three-dimensional image result; The device for pipeline interior three-dimensional structure detection and imaging is composed of a pair of pattern projection lamps, a multi-camera combination, a mechanical connecting rod, a translation table, a computer and a stereo chessboard target, one end of the mechanical connecting rod is connected with the translation table, and the front part of the other end is provided with the pattern projection lamp (1), the multi-camera combination and the pattern projection lamp (2) along the axial direction thereof, and the computer is connected with the pair of pattern projection lamps, the multi-camera combination and the translation table respectively. The stereo chessboard target is a hollow four-sided column with a square cross section, two opposite square surfaces are open, and chessboard targets are made on the four inner surfaces.

2. The information processing method of a device for internal three-dimensional structure detection and imaging of a pipe according to claim 1, characterized by, The detection process of step one comprises the following steps: S1: multi-camera combination calibration using a stereo chessboard target; S2: replacing the stereo chessboard target with the pipeline to be measured, and the center of the pipeline coincides with the axis of the connecting rod; S3: using the computer to control the pattern projection lamp to project a random point solid image on the inner wall of the pipeline, and using the multi-camera combination to simultaneously acquire projection images of different angles inside the pipeline; then controlling the pattern projection lamp to uniformly illuminate the inner wall of the pipeline, and using the multi-camera combination to acquire the illumination image of the inner wall of the pipeline; S4: controlling the pattern projection lamp and the multi-camera combination to move inside the pipeline, and repeating S3 at different depth positions until the pipeline is completely detected.

3. The information processing method of a device for detecting and imaging a three-dimensional structure inside a pipe according to claim 1 or 2, characterized by, The specific steps of step two are: the process of three-dimensional structure reconstruction of the pipeline interior using the projection image comprises the following steps: S1: detecting and matching feature points on the projection random point solid images acquired by the adjacent two cameras to obtain the image coordinate system positions of the matched projection feature point pairs in the two projection images; S2: preprocessing the projection feature point pairs to eliminate obviously mismatched feature point pairs; S3: any of the remaining feature points, which are imaged in the i-th and i+1-th cameras, with image coordinate systems in the two projections (x' i ,y' i ) and (x' i+1 ,y' i+1 ) respectively, focal lengths f i and f i+1 , rotation matrix R and translation vector T respectively, World coordinate system (x w ,y w ,z w ) and image coordinate system (x' i ,y' i ), (x' i+1 ,y' i+1 ) of the feature point satisfy the relationship: The above system of equations is established using the image coordinate system of the feature point and the camera parameters. By solving the system of equations, the world coordinate system position (x, y) of the feature point can be obtained. w ,y w ,z w ); S4: obtaining the world coordinate system positions of all feature points by using S3, and then performing curve fitting to obtain the three-dimensional structure of the pipeline interior.

Images