Binocular full-angle laser three-dimensional scanning device and method for interior of narrow pipeline

Abstract

The invention discloses a binocular full-angle laser three-dimensional scanning device and method for the interior of a narrow pipeline. The device comprises a Mecanum main driving wheel, a binocular camera, a full-angle laser transmitter and a driven wheel. According to the invention, after the device enters a pipeline to move at a constant speed and a binocular annular laser scanning binocular image sequence is obtained to be corrected and corrected, a laser stripe segmentation result is obtained through a GuideNet-based laser stripe segmentation method, and a binocular annular laser center line is extracted based on a Steger algorithm, so that a laser stripe segmentation result is obtained. And extracting a laser center point of the annular middle section area and performing three-dimensional reconstruction, fitting a transient laser plane by using the point cloud of the part, fitting a full-angle annular laser ring three-dimensional point cloud of the current image frame based on the transient laser plane, and finally realizing splicing of the laser point cloud corresponding to each frame of image by combining the movement speed and the image frame rate. And three-dimensional reconstruction of the inner surface of the pipeline is realized. Accurate data can be provided for pipeline deformation detection, and the method has high engineering application value.

Description

Technical Field

[0001] The present invention relates to a binocular full-angle laser three-dimensional scanning device and method for the interior of narrow pipes, belonging to the technical fields of three-dimensional laser scanning, computer vision, and optical measurement. Background Art

[0002] The technical background of the binocular full-angle laser three-dimensional scanning device and method for the interior of narrow pipes mainly revolves around the complexity of pipe inspection in fields such as industry, petroleum, chemical engineering, and municipal engineering. Pipes play a crucial role in these fields. However, with long-term use, problems such as corrosion, cracks, sedimentation, and deformation are likely to occur inside the pipes. Traditional inspection methods such as mechanical inspection, acoustic inspection, and endoscope inspection often cannot achieve comprehensive and efficient inspection due to factors such as narrow pipe space, complex structure, and pipe bending. Especially when facing long-distance or complex-structured pipes, the limitations of these traditional methods become more obvious.

[0003] Three-dimensional laser scanning technology has made significant progress in large-scale measurement applications such as building and terrain mapping. However, due to the large volume and complex structure of traditional laser scanning equipment, it cannot adapt to narrow pipe spaces, resulting in limited scanning perspectives and an inability to comprehensively cover the inner wall of the pipe. At the same time, binocular vision technology obtains three-dimensional information by simulating the working principle of human eyes and using the parallax principle, and is widely used in fields such as robot navigation, autonomous driving, and industrial inspection. However, in a narrow pipe environment, the binocular vision system is easily affected by factors such as insufficient light and limited perspectives, making it difficult to achieve high-precision panoramic three-dimensional scanning.

[0004] Full-angle scanning is crucial for pipe inspection, especially inside narrow pipes. Traditional inspection methods such as push-type endoscopes can partially complete the inspection, but there are still blind spots and they cannot cover all the inner walls of the pipes, possibly missing important defect information. To overcome these technical bottlenecks, a full-angle three-dimensional scanning device that combines laser scanning and binocular vision technology has become an ideal solution. Laser scanning technology can provide high-precision depth information to complement the deficiencies of binocular vision under complex lighting conditions, while binocular vision can provide surface texture and structure information through image processing. The combination of the two significantly improves the inspection efficiency and accuracy in a narrow pipe environment, meeting the need for comprehensive and non-blind-spot three-dimensional inspection of the interior of the pipe. Summary of the Invention

[0005] Object of the Invention: The technical problem to be solved by the present invention is to propose a binocular full-angle laser three-dimensional scanning device and method for the interior of narrow pipes, overcome the limitations in the narrow and complex pipe environment in the prior art, and achieve comprehensive, efficient, and accurate inspection of the interior of the pipes.

[0006] The above object is achieved by the following technical solutions:

[0007] The present invention first provides a binocular full-angle laser three-dimensional scanning device inside a narrow pipeline. The device includes front and rear device support frames connected together by a connecting rod. The front and rear device support frames are respectively connected to a Mecanum main drive wheel and a driven wheel. The Mecanum main drive wheel and the driven wheel can be controlled by a remote control to perform forward and backward operations. A binocular camera and a full-angle laser emitter are respectively installed on the front and rear device support frames; the binocular camera and the full-angle laser emitter are installed facing each other. And during the process of the device moving in the pipeline, the full-angle laser emitter is used to project laser stripes in all directions and form an annular laser pattern inside the pipe wall, and at the same time, it is captured and imaged by the binocular camera to obtain a binocular image sequence containing annular laser stripes, and then it is transmitted to a data receiving and processing platform in real time through a power supply + data transmission line for analysis and processing.

[0008] The present invention also provides a method for binocular full-angle laser three-dimensional scanning inside a narrow pipeline using the above-mentioned binocular full-angle laser three-dimensional scanning device inside a narrow pipeline. The method includes the following steps:

[0009] (1) Calibrate the binocular camera. The binocular camera includes a left-eye camera and a right-eye camera, and obtain the internal parameter K l of the left-eye camera, the distortion coefficient Distortion l of the left-eye camera, the internal parameter K r of the right-eye camera, the distortion coefficient Distortion r of the right-eye camera, the translation matrix T rl between the binocular cameras, the rotation distance R rl between the binocular cameras, where the baseline distance between the binocular cameras is Baseline;

[0010]

[0011] Among them, represents the x-axis focal length of the left-eye camera, represents the y-axis focal length of the left-eye camera, represents the x-axis coordinate of the principal point in the coordinate system of the left-eye camera, represents the y-axis coordinate of the principal point in the coordinate system of the left-eye camera, represents the x-axis focal length of the right-eye camera, represents the y-axis focal length of the right-eye camera, represents the x-axis coordinate of the principal point in the coordinate system of the right-eye camera, represents the y-axis coordinate of the principal point in the coordinate system of the right-eye camera;

[0012] (2) Use the binocular full-angle laser three-dimensional scanning device to move uniformly inside the pipeline, and the movement speed is denoted as move v , and use the binocular camera to collect the binocular annular laser image sequence Images k = {I L (k), I R (k)}, where I L (k) represents the original image frame collected by the left camera, I R (k) represents the original image frame collected by the right camera, k represents the k-th frame of the image, and the frame rate of the binocular camera is denoted as fps cam , the height of both the left-eye image and the right-eye image is H img , and the width of both is W img , and transmit them to the processing platform through the signal line for processing; use the internal parameters of the left camera, the internal parameters of the right camera, the translation matrix, and the rotation matrix between the binocular cameras to correct and calibrate the binocular annular laser image sequence Images k = {I L (k), I R (k)} to obtain a new binocular image sequence ImagesC k = {Ic L (k), Ic R (k)}, where Ic L (k) represents the left-eye image after correction and calibration processing, and Ic R (k) represents the right-eye image after correction and calibration processing of the right eye;

[0013] (3) Adopt the laser stripe segmentation method based on GuideNet to quickly segment the annular laser stripe regions in the new binocular image sequence ImagesC k = {Ic L (k), Ic R (k)} obtained in step (2) respectively, and obtain the left-eye annular laser stripe segmentation map Seg l and the right-eye annular laser stripe segmentation map Seg r ;

[0014] (4) Use the binocular annular laser centerline extraction method based on the Steger algorithm to extract the centerlines inside the left-eye image annular laser stripe segmentation map and the right-eye annular laser stripe segmentation map respectively, and obtain the set of laser stripe center points in the left-eye image and the set of laser stripe center points in the right-eye image Among them, Mc l , Mc r respectively represent the total number of center points detected in the left-eye image and the right-eye image; represents the image coordinates of the i-th left-eye laser center point, denote the image coordinates of the j-th right-eye laser center point;

[0015] (5) Conduct 3D reconstruction of the laser center points in the annular middle section area: Screen the laser stripe center points in the annular middle section area from the left-eye annular laser stripe segmentation map Seg l and the right-eye annular laser stripe segmentation map Seg r to obtain the set of laser stripe center points in the annular middle section area in the left-eye image and the set of laser stripe center points in the annular middle section area in the right-eye image where Mcc l , Mcc r respectively represent the total number of laser stripe center points in the annular middle section area in the left-eye image and the right-eye image, and use the limit constraint principle and color similarity to process the set of laser stripe center points PCC l in the annular middle section area in the left-eye image and the set of laser stripe center points PCC r in the annular middle section area in the right-eye image for matching to determine the set of optimal matching point pairs i mp = 1, 2, 3, …, M match , M match represents the total number of optimal matching points; Calculate the 3D coordinates of the pixel points l in Seg according to the set of optimal matching point pairs MatchP where the 3D coordinates of the i mp -th point are respectively representing the 3D coordinates of the x-axis, y-axis, and z-axis;

[0016] (6) Fit the transient laser plane equation: Use the general form A P X + B P Y + C P Z + D P = 0 to represent the transient laser light plane equation at the current moment, where X, Y, Z represent the coordinate variables in 3D space, and use the unconstrained optimization simplex method (Nelder-Mead Algorithm) to combine to fit and solve the parameters {A P , B P , C P , D P} in the general form to obtain the transient laser light plane equation at the current moment;

[0017] (7) 3D reconstruction of the full-angle annular laser ring based on transient laser plane fitting: Using the plane equation parameters, establish the transient laser light plane equation and the internal parameters of the left eye camera to realize the 3D reconstruction of all the center points of the annular laser in the left eye image of the k-th frame image;

[0018] (8) Stitch the annular laser scanning data obtained at each moment according to the moving distance to realize the reconstruction of the 3D point cloud data inside the pipeline Where Num is the total number of frames of the collected images.

[0019] Furthermore, the specific implementation steps of the laser stripe segmentation method based on GuideNet in step (3) are as follows:

[0020] Taking the corrected and calibrated left eye image Ic L (k) as an example,

[0021] (3-1) Preprocess the input image: Convert the original laser stripe image into a grayscale image I gray , and perform normalization processing I norm represents the image after normalizing the grayscale image I gray , and scale it to the standard size H×W: I input = resize(I norm , (H,W)), I input represents the input image of the laser stripe segmentation method based on GuideNet, and resize(I norm , (H,W)) represents the operation of adjusting the size of I norm to the scaled standard size H×W;

[0022] (3-2) Input I input into the initial convolutional layer to extract initial features;

[0023] (a) Use three-branch parallel convolution for the input image I input , perform normalization respectively and then add them for feature fusion, and use the ReLU activation function for activation to obtain the fused feature F mer in the first stage:

[0024] F merg = ReLU(BN(Conv 3×3,S2 (I input )) + BN(Conv 3×1,S2 (I input )) + BN(Conv 1×3,S2 (I input ))) (2)

[0025] Among them, BN(·) represents normalization processing, ReLU(·) represents the ReLU activation function, and Conv 3×3,S2 (·) represents convolution processing using a 3×3 convolution kernel with a stride of 2, and Conv 3×1,S2 (·) represents convolution processing using a 3×1 convolution kernel with a stride of 2, and Conv 1×3,S2 (·) represents convolution processing using a 1×3 convolution kernel with a stride of 2;

[0026] (b) Use a 3×3 depth convolution B with a stride of 2 3×3,S2 (·) and a 1×1 convolution kernel Conv with a stride of 2 1×1,S2 (·) to perform convolution on the fused feature F in the first stage merge1 respectively, add and fuse the results after convolution, then perform dimensionality reduction processing using a 1×1 convolution kernel with a stride of 2, and finally activate using the ReLU activation function to obtain the initial feature map F conv :

[0027] F conv = ReLU(Conv 1×1,S2 (DWConv 3×3,S2 (F merge1 ) + Conv 1×1,S2 (F merge1 ))) (3)

[0028] (3-2) Input the preliminary feature map F conv into the dual-branch encoder to extract high-level semantic features and high-resolution spatial information respectively;

[0029] (a) Input the preliminary feature map F conv into the semantic feature extraction branch, and use 4 cascaded Transformer feature extraction blocks to obtain four levels of high-level semantic feature maps i sem = {1, 2, 3, 4}:

[0030]

[0031] Among them, MHSA(·) represents multi-head self-attention processing, and FFN(·) represents feed-forward neural network processing;

[0032] (b) Input F conv and the four levels of high-level semantic feature maps into the spatial feature extraction branch for processing. The spatial feature extraction branch is composed of 4 spatial feature extraction blocks and a guiding module cross-cascaded. Taking the i semTaking the combination of {1, 2, 3, 4} spatial feature extraction blocks and the guiding module as an example: Assume the input of the spatial feature extraction block is and the output is Then, it is jointly input into the guiding module with the high-level semantic feature map of each layer for feature fusion to obtain the fused guiding feature

[0033] First, calculate the depthwise separable convolutional feature map

[0034]

[0035] where DSC(·) represents the depthwise separable convolutional operation, DWConv 3×3,S1 (·) represents the depth convolution with a size of 3×3 and a stride of 1, and PWConv 3×3,S1 (·) represents the pointwise convolution processing with a size of 3×3 and a stride of 1;

[0036] Perform normalization and ReLU activation processing on to obtain: Respectively use the convolution Conv 3×1,S1 (·) with a size of 3×3 and a stride of 1, the convolution Conv 1×3,S1 (·) with a size of 1×3 and a stride of 1, and the convolution Conv 3×1,S1 (·) with a size of 3×1 and a stride of 1 to perform convolution processing on Then, normalize and add them respectively, and after ReLU activation, obtain the spatial feature map

[0037]

[0038] (c) Using the guiding module, fuse the semantic feature map of each layer and the spatial feature map output by the spatial feature extraction block to obtain the fused guiding feature First, calculate the semantic feature map and the spatial feature map of the similarity map

[0039]

[0040] where σ(·) represents the Sigmoid activation function, UP(·) represents the upsampling process of the semantic feature map to make its shape consistent with that of the spatial feature map, and ⊙ represents element-wise multiplication;

[0041] Based on the similarity map calculate the fused feature map

[0042]

[0043] Further calculate the topological attention gating value

[0044]

[0045] Among them, IN(·) represents instance normalization, and DSConv(·) represents dynamic convolution based on topological geometric constraints. represents the similarity-guided weighted fusion feature of semantic features and spatial features;

[0046] Utilize the topological attention gating value to multiply with the fusion feature map pixel by pixel to obtain the fusion-guided feature

[0047]

[0048] (3-3) Input the fusion-guided feature maps of different layers into the multi-level feature fusion decoder, fuse the multi-level features, and gradually restore them to the original resolution to obtain the final laser stripe segmentation result:

[0049] (a) First, expand the channels of to obtain the expanded fusion-guided feature map Calculate the attention weight of the i-th layer Fuse the multi-level features by weighted summation layer by layer:

[0050]

[0051] Among them, represents the highest-level feature map, and W 4 represents the weight value of the highest-level feature map;

[0052] (b) Set the threshold τ seg , and obtain the final binary segmentation result Seg l of the left-eye image:

[0053]

[0054] (3-4) Input the corrected and calibrated right-eye image Ic R (k), and use the method in steps (3-1)-(3-3) to obtain the final binary segmentation result Seg r of the right-eye image.

[0055] Further, the specific implementation steps of performing three-dimensional reconstruction of the laser center point in the annular middle section area in step (5) are as follows:

[0056] (5-1) Assume that in the segmented map Seg of the annular laser stripe in the left-eye image l to obtain the set of laser stripe center points in the left-eye image The row label of the first row that appears is start cent , and the last row is end cent , that is, the set of laser stripe center points in the left-eye image The x-axis coordinate range where it appears is [start cent , end cent . Select the points whose row label range falls within [start cent +Thre cent , end cent -Thre cent as the set of laser stripe center points in the left-eye image located in the annular middle section area. Among them, Similarly, select the center points in the right-eye image whose row label falls within [start cent +Thre cent , end cent -Thre cent as the set of laser stripe center points in the right-eye image located in the annular middle section area; after screening, denote the set of center points in the left-eye image located in the annular middle section area as Denote the set of laser stripe center points in the right-eye image located in the annular middle section area as Among them, Mcc l , Mcc r respectively represent the total number of laser stripe center points in the left-eye image and the right-eye image located in the annular middle section area;

[0057] (5-2) For each laser center point in the annular middle section area of the left-eye image Calculate its color feature Among them, round(·) represents rounding to the nearest integer. According to the epipolar constraint, screen the right-eye candidate points. In the set of center points in the right-eye image located in the annular middle section area , find the points that satisfy the epipolar constraint for the row coordinate and whose column coordinate is on the

[0058] left side of and add them to the matching candidate point set Cand:

[0059] Assume that a total of J are foundc one matching candidate points, calculate the color feature of the j-th point in the matching candidate point set Cand c : j c = 1, 2, …, J c ; Calculate the color feature difference between the candidate matching points and find the index corresponding to the minimum distance to obtain the matching point Cand(j * );

[0060] (5 - 3) Calculate the matching point pairs of each laser center point in the middle annular section area of the left-eye image in sequence, and obtain the optimal matching point pair set M match represents the total number of optimal matching point pairs;

[0061] (5 - 4) Use the matching point pairs MatchP located in the middle annular section area as reliable matching point pairs for fitting the laser light plane, and combine the baseline distance and the left-eye camera parallax to realize the three-dimensional reconstruction of this part of the point cloud MatchP 3D , assuming i mp = 1, 2, 3, …, M match , and respectively represent and 's column coordinates, and respectively represent and 's row coordinates, calculate the parallax then the three-dimensional coordinates of the matching points in the middle annular section area of the left-eye image can be calculated

[0062]

[0063] Furthermore, the specific implementation steps of the three-dimensional reconstruction of the full-angle annular laser ring based on transient laser plane fitting described in step (7) thereof are as follows:

[0064] (7 - 1) Convert the image coordinates of the laser stripe center point set in the left-eye image to the normalized camera coordinates X n :

[0065]

[0066] Among them, and respectively represent the x-axis coordinate and y-axis coordinate of the -th point in the normalized camera coordinate system, and K l represents the internal parameters of the left-eye camera;

[0067] (7-2) Calculate the three-dimensional coordinates of the center point of the ring laser using the transient laser light plane equation:

[0068] Define the light plane parameter vector A = [A P B P , and define the X and Y coordinates in the normalized camera coordinate: where (X ni , Y ni ) is the camera coordinate after pixel point normalization, and calculate the depth value Z(k) of all center points of the ring laser in the k-th frame image:

[0069]

[0070] Calculate the three-dimensional coordinates XY(k) of the X-axis and Y-axis of all center points of the ring laser in the k-th frame image:

[0071]

[0072] where, ⊙ represents element-wise multiplication;

[0073] (7-3) Then the final three-dimensional coordinates P 3D (k) of all center points of the ring laser in the k-th frame image:

[0074]

[0075] where, fps cam represents the frame rate of the binocular camera, and move v represents the moving speed of the binocular full-angle laser three-dimensional scanning device moving uniformly inside the pipeline.

[0076] Compared with the prior art, the beneficial effects of the present invention are:

[0077] (1) By combining the binocular vision system with the full-angle laser scanning technology, the present invention can achieve a full-range scan of the inside of the pipeline, avoiding the dead angle problem that may exist in traditional detection equipment, and ensuring a comprehensive inspection and accurate monitoring of the inner wall of the pipeline.

[0078] (2) By extracting the center line position of the laser stripe using the Steger algorithm and combining the image data captured by the binocular camera, the present invention can generate a high-precision three-dimensional point cloud data model, accurately reflecting the geometric shape and defects inside the pipeline.

[0079] (3) The present invention adopts a Mecanum wheel drive system, which has the ability of omnidirectional movement, can flexibly cope with the narrow space and complex bending structure in the pipeline, realizes the efficient movement and precise positioning of the detection device, and is suitable for use in various complex environments.

[0080] (4) The present invention can combine the real-time movement of the device, quickly collect and splice three-dimensional data from multiple perspectives, significantly improve the efficiency of pipeline detection, reduce the detection time and manual intervention, and is applicable to the daily monitoring and maintenance of large-scale pipeline systems. Description of the Drawings

[0081] Figure 1 It is a schematic structural diagram of a binocular full-angle laser three-dimensional scanning device inside a narrow pipeline of the present invention;

[0082] Figure 2 It is a flowchart of the method of the present invention.

[0083] Figure 3 It is a network framework diagram of the laser stripe segmentation method based on GuideNet.

[0084] Figure 4 It is a block diagram of the initial convolutional layer in the laser stripe segmentation method based on GuideNet.

[0085] Figure 5 It is a block diagram of the spatial feature extraction block in the laser stripe segmentation method based on GuideNet.

[0086] Figure 6 It is a block diagram of the guiding module in the laser stripe segmentation method based on GuideNet.

[0087] Figure 7 It is a schematic diagram of the method for extracting the center point of the annular middle section area of the laser stripe pattern. Detailed Embodiments

[0088] The binocular full-angle laser three-dimensional scanning device and method inside a narrow pipeline in the present invention are as follows Figure 1As shown in the figure. The device includes a Mecanum main drive wheel 1, a binocular camera 2, a full-angle laser emitter 3, a device support frame 4, a connecting rod 5, a power supply + data transmission line 6, a data receiving and processing platform 7, and a driven wheel 8. The device can control the Mecanum main drive wheel 1 on the device to move forward and backward through a remote control. Among them, the device support frame 4 is used to support the binocular full-angle laser three-dimensional scanning device, helping the 1-Mecanum main drive wheel 1 and the driven wheel 8 to better fit the pipe wall. The connecting rod 5 is used to fix the front and rear parts of the full-angle laser three-dimensional scanning device. The driven wheel 8 is used to assist the 1-Mecanum main drive wheel 1 to achieve stable movement of the device in the pipe. The binocular camera 2 and the full-angle laser 3 are installed facing each other. During the movement of the device in the pipe, the full-angle laser emitter 3 projects laser stripes in all directions and forms an annular laser pattern inside the pipe wall, which is simultaneously captured by the binocular camera 2 to obtain a binocular image sequence containing annular laser stripes, and then transmitted to the data receiving and processing platform 7 in real time through the power supply + data transmission line 6 for analysis and processing.

[0089] The method for performing binocular full-angle laser three-dimensional scanning inside a narrow pipe using the above-mentioned binocular full-angle laser three-dimensional scanning device inside a narrow pipe is as follows Figure 2 shown: The method includes the following steps:

[0090] (1) Calibrate the binocular camera 2. The binocular camera 2 includes a left-eye camera and a right-eye camera, and obtain the internal parameters K l of the left-eye camera, the distortion coefficient Distortion l of the left-eye camera, the internal parameters K r of the right-eye camera, the distortion coefficient Distortion r of the right-eye camera, the translation matrix T rl between the binocular cameras, and the rotation distance R rl between the binocular cameras, where the baseline distance between the binocular cameras is Baseline;

[0091]

[0092] Among them, represents the x-axis focal length of the left-eye camera, represents the y-axis focal length of the left-eye camera, represents the x-axis coordinate of the principal point in the left-eye camera coordinate system, represents the y-axis coordinate of the principal point in the left-eye camera coordinate system, represents the x-axis focal length of the right-eye camera, represents the y-axis focal length of the right-eye camera, represents the x-axis coordinate of the principal point in the right-eye camera coordinate system, represents the y-axis coordinate of the principal point in the right-eye camera coordinate system;

[0093] (3) Use a binocular full-angle laser 3D scanning device to move uniformly inside the pipeline, and record the movement speed as move v , and use a binocular camera to collect a sequence of binocular annular laser images Images k = {I L (k), I R (k)}, where I L (k) represents the original image frame collected by the left camera, I R (k) represents the original image frame collected by the right camera, k represents the k-th frame image, and the frame rate of the binocular camera is recorded as fps cam , the height of both the left-eye image and the right-eye image is H img , and the width of both is W img , and transmit them to the processing platform through signal lines for processing; use the internal parameters of the left camera, the internal parameters of the right camera, the translation matrix, and the rotation matrix between the binocular cameras to correct and calibrate the sequence of binocular annular laser images Images k = {I L (k), I R (k)} to obtain a new sequence of binocular images ImagesC k = {Ic L (k), Ic R (k)}, where Ic L (k) represents the left-eye image after correction and calibration processing, and Ic R (k) represents the right-eye image after correction and calibration processing of the right eye;

[0094] (3) As shown in Figure 3 , adopt a laser stripe segmentation method based on GuideNet to quickly segment the annular laser stripe regions in the new sequence of binocular images ImagesC k = {Ic L (k), Ic R (k)} obtained in step (2) respectively, and obtain the left-eye annular laser stripe segmentation map Seg l and the right-eye annular laser stripe segmentation map Seg r ;

[0095] (4) Use a binocular annular laser centerline extraction method based on the Steger algorithm to extract the centerlines inside the left-eye image annular laser stripe segmentation map and the right-eye image annular laser stripe segmentation map respectively, and obtain the set of laser stripe center points in the left-eye image and the set of laser stripe center points in the right-eye image where, Mc l , Mc r respectively represent the total number of center points detected in the left-eye image and the right-eye image; represents the image coordinates of the center point of the i-th left-eye laser represents the image coordinates of the center point of the j-th right-eye laser;

[0096] (5) Conduct 3D reconstruction of the laser center points in the annular middle section area: Screen the center points of the laser stripes in the annular middle section area in the left-eye annular laser stripe segmentation map Seg l and the right-eye annular laser stripe segmentation map Seg r to obtain the set of center points of the laser stripes in the annular middle section area in the left-eye image and the set of center points of the laser stripes in the annular middle section area in the right-eye image where, Mcc l , Mcc r respectively represent the total number of center points of the laser stripes in the annular middle section area in the left-eye image and the right-eye image, and using the principle of limit constraint and color similarity, for the set of center points of the laser stripes in the annular middle section area in the left-eye image PCC l and the set of center points of the laser stripes in the annular middle section area in the right-eye image PCC r perform matching to determine the set of best matching point pairs i mp = 1, 2, 3, …, M match , M match represents the total number of best matching points; Calculate the 3D coordinates of the pixel points l in Seg in the point pairs according to the set of best matching point pairs MatchP where, the 3D coordinates of the i mp -th point are respectively representing the 3D coordinates of the x-axis, y-axis, and z-axis;

[0097] (6) Fit the transient laser plane equation: Use the general form A P X + B P Y + C P Z + D P = 0 to represent the transient laser light plane equation at the current moment, where X, Y, Z represent the coordinate variables in 3D space, and use the unconstrained optimization simplex method (Nelder-Mead Algorithm), combined with fit and solve for the parameters {A P , B P , C P , D P}, obtain the transient laser light plane equation at the current moment;

[0098] (7) Three-dimensional reconstruction of the full-angle annular laser ring based on transient laser plane fitting: Using the plane equation parameters, establish the transient laser light plane equation and the internal parameters of the left-eye camera, and realize the three-dimensional reconstruction of all the annular laser center points in the left-eye image of the k-th frame image ;

[0099] (8) Stitch the annular laser scan data obtained at each moment according to the moving distance to realize the reconstruction of the three-dimensional point cloud data inside the pipeline where Num is the total number of frames of the collected images.

[0100] The specific implementation steps of the laser stripe segmentation method based on GuideNet described in step (3) of this embodiment are as follows:

[0101] Taking the corrected and calibrated left-eye image Ic L (k) as an example,

[0102] (3-1) Preprocess the input image: Convert the original laser stripe image into a grayscale image I gray , and perform normalization processing I norm represents the image after normalizing the grayscale image I gray , and scale it to the standard size H×W: I input = resize(I norm , (H,W)), i input represents the input image of the laser stripe segmentation method based on GuideNet, and resize(I norm , (H,W)) represents the operation of adjusting the size of I norm to the scaled standard size H×W;

[0103] (3-2) As Figure 4 shown, input I input into the initial convolutional layer to extract the initial features;

[0104] (a) Use three-branch parallel convolution for the input image I input , perform normalization processing on each branch and then add them for feature fusion, and use the ReLU activation function for activation to obtain the fused feature F merge1 in the first stage:

[0105] F merge1 = ReLU(BN(Conv 3×3,S2 (I input )) + BN(Conv 3×1,S2 (I input )) + BN(Conv1×3,S2 (I input ))) (2)

[0106] Among them, BN(·) represents normalization processing, ReLU(·) represents the ReLU activation function, and Conv 3×3,S2 (·) represents convolution processing using a 3×3 convolution kernel with a stride of 2, and Conv 3×1,S2 (·) represents convolution processing using a 3×1 convolution kernel with a stride of 2, and Conv 1×3,S2 (·) represents convolution processing using a 1×3 convolution kernel with a stride of 2;

[0107] (b) Using a 3×3 depthwise convolution DWConv 3×3,S2 (·) and a 1×1 convolution kernel with a stride of 2, Conv 1×1,S2 (·) respectively perform convolution on the fused feature F merge1 in the first stage, add and fuse the results after convolution, then perform dimensionality reduction processing using a 1×1 convolution kernel with a stride of 2, and finally use the ReLU activation function for activation to obtain the initial feature map F conv :

[0108] F conv = ReLU(Conv 1×1,S2 (DWConv 3×3,S2 (F merge1 ) + Conv 1×1,S2 (F merge1 ))) (3)

[0109] (3 - 2) Input the preliminary feature map F conv into the dual-branch encoder to extract high-level semantic features and high-resolution spatial information respectively;

[0110] (a) Input the preliminary feature map F conv into the semantic feature extraction branch, and use 4 cascaded Transformer feature extraction blocks to obtain four levels of high-level semantic feature maps i sem = {1, 2, 3, 4}:

[0111]

[0112] Among them, MHSA(·) represents multi-head self-attention processing, and FFN(·) represents feed-forward neural network processing;

[0113] (b) As Figure 5 shown, input F conv and the four levels of high-level semantic feature maps The input space feature extraction branch processes. The space feature extraction branch is composed of cross-cascading 4 space feature extraction blocks and a guiding module. Taking the combination of the i sem ={1, 2, 3, 4} space feature extraction blocks and the guiding module as an example: Suppose the input of the space feature extraction block is and the output is Then it is jointly input into the guiding module with the high-level semantic feature map of each layer for feature fusion to obtain the fused guiding feature

[0114] First, calculate the depthwise separable convolution feature map

[0115]

[0116] where DSC(·) represents the depthwise separable convolution operation, and DWConv 3×3,S1 (·) represents a 3×3 depth convolution with a stride of 1, and PWConv 3×3,S1 (·) represents a 3×3 pointwise convolution with a stride of 1;

[0117] For perform normalization and ReLU activation processing to obtain: Respectively use a 3×3 convolution Conv 3×1,S1 (·), a 1×3 convolution Conv 1×3,S1 (·), and a 3×1 convolution Conv 3×1,S1 (·) to perform convolution processing on , then normalize and add them respectively, and after ReLU activation, obtain the space feature map

[0118]

[0119] (c) As Figure 6 shown, use the guiding module to fuse the semantic feature map of each layer and the space feature map output by the space feature extraction block to obtain the fused guiding feature First, calculate the similarity map of the semantic feature map and the space feature map

[0120]

[0121] Among them, σ(·) represents the Sigmoid activation function, UP(·) represents the upsampling process of the semantic feature map to make it consistent with the shape of the spatial feature map, and ⊙ represents element-wise multiplication;

[0122] Based on the similarity graph Calculate the fused feature map

[0123]

[0124] Further calculate the topological attention gating value

[0125]

[0126]

[0127] Among them, IN(·) represents instance normalization, and DSConv(·) represents dynamic convolution based on topological geometric constraints. Represents the similarity-guided weighted fusion feature of semantic features and spatial features;

[0128] Utilize the topological attention gating value And the fused feature map Perform element-wise multiplication to obtain the fusion-guided feature

[0129]

[0130] (3-3) Input the fusion-guided feature maps of different layers Into the multi-level feature fusion decoder, fuse the multi-level features, and gradually restore them to the original resolution to obtain the final laser stripe segmentation result:

[0131] (a) First, Perform channel expansion to obtain the expanded fusion-guided feature map Calculate the attention weight of the i-th layer Through layer-by-layer weighted summation, fuse the multi-level features:

[0132]

[0133] Among them, Represents the highest-level feature map, and W 4 Represents the weight value of the highest-level feature map;

[0134] (b) Set the threshold τ seg , in the present invention, take the empirical value τ seg =0.9 to obtain the final binary segmentation result Seg l :

[0135]

[0136] (3 - 4) Right eye image Ic after input correction and calibration processing R (k), using the method of steps (3 - 1)-(3 - 3), obtain the final binary segmentation result Seg of the laser stripe of the right eye image r .

[0137] The specific implementation steps of performing 3D reconstruction of the laser center points in the annular middle section area described in step (5) of this embodiment are as follows:

[0138] (5 - 1) As Figure 7 shown, assuming in the annular laser stripe segmentation map Seg l of the left eye image, obtain the set of laser stripe center points in the left eye image The row label of the first row where it appears is start cent , and the last row is end cent , that is, the set of laser stripe center points in the left eye image The x - axis coordinate range where it appears is [start cent , end cent . Select the points whose row label range falls within [start cent + Thre cent , end cent - Thre cent as the set of laser stripe center points in the annular middle section area of the left eye image. Among them, Similarly, select the center points whose row label in the right eye image falls within [start cent + Thre cent , end cent - Thre cent as the set of laser stripe center points in the annular middle section area of the right eye image; after screening, denote the set of center points in the annular middle section area of the left eye image as Denote the set of laser stripe center points in the annular middle section area of the right eye image as Among them, Mcc l , Mcc r respectively represent the total number of laser stripe center points in the annular middle section area of the left eye image and the right eye image.

[0139] (5 - 2) For each laser center point in the annular middle section area of the left eye image Calculate its color feature Among them, round(·) represents rounding to the nearest integer. According to the epipolar constraint, the right-eye candidate points are screened, and the set of center points located in the middle section of the ring in the right-eye image find the points that satisfy the epipolar constraint for the row coordinate and the column coordinate is in the left, and add these points to the matching candidate point set Cand:

[0140]

[0141] Among them, δ is the threshold for the row coordinate constraint. In the present invention, the empirical value δ = 2 is taken. If there are no matching candidate points, the matching is abandoned

[0142] Suppose a total of J c points of matching candidate points are found, and calculate the color feature of the j c -th point in the matching candidate point set Cand: j c = 1, 2, …, J c ; calculate the color feature difference between and the candidate matching point, and find the index corresponding to the minimum distance to obtain the matching point Cand(j * ).

[0143] (5 - 3) Calculate the matching point pairs of each laser center point in the middle section of the ring in the left-eye image in turn to obtain the set of best matching point pairs M match represents the total number of best matching point pairs.

[0144] (5 - 4) Take the matching point pairs MatchP located in the middle section of the ring as the reliable matching point pairs for fitting the laser light plane, and combine the baseline distance and the left-eye camera parallax to realize the three-dimensional reconstruction of this part of the point cloud MatchP 3D Suppose i mp = 1, 2, 3, …, M match , and respectively represent and the column coordinates of and respectively represent and the row coordinates of calculate the parallax

[0145]

[0146] In step (7) of this embodiment, the specific implementation steps of the three-dimensional reconstruction of the full-angle annular laser ring based on transient laser plane fitting are as follows:

[0147] (7-1) Convert the image coordinates of the set of laser stripe center points in the left-eye image to the normalized camera coordinates X n :

[0148]

[0149] wherein, and respectively represent the x-axis coordinate and y-axis coordinate of the th point in the normalized camera coordinate system, and K l represents the internal parameters of the left-eye camera.

[0150] (7-2) Calculate the three-dimensional coordinates of the annular laser center point by using the transient laser light plane equation:

[0151] Define the light plane parameter vector A = [A P B P , and define the X and Y coordinates in the normalized camera coordinates: where is the camera coordinate after pixel normalization, and calculate the depth value Z(k) of all annular laser center points in the k-th frame image:

[0152]

[0153] Calculate the three-dimensional coordinates XY(k) of the X-axis and Y-axis of all annular laser center points in the k-th frame image:

[0154]

[0155] wherein, ⊙ represents element-wise multiplication.

[0156] (7-3) Then the final three-dimensional coordinates P 3D (k) of all annular laser center points in the k-th frame image:

[0157]

[0158] wherein, fps cam represents the frame rate of the binocular camera, and move v represents the moving speed of the binocular full-angle laser three-dimensional scanning device moving uniformly inside the pipeline.

Claims

1. A binocular full-angle laser three-dimensional scanning device inside a narrow pipe, characterized in that: The device comprises a front and rear device support frame (4) connected together by a connecting rod (5), the front and rear device support frames (4) are respectively connected to a Mecanum main drive wheel (1) and a driven wheel (8), the Mecanum main drive wheel (1) and the driven wheel (8) can be controlled by a remote controller to move forward and backward, and the front and rear device support frames (4) are respectively installed with a binocular camera (2) and a full-angle laser transmitter (3); the binocular camera (2) and the full-angle laser transmitter (3) are installed opposite to each other, and when the device is moving in a pipeline, the full-angle laser transmitter (3) is used to project laser stripes to the surroundings, and form an annular laser pattern inside the pipe wall, and is simultaneously photographed by the binocular camera (2) to obtain a binocular image sequence containing the annular laser stripes, and then transmitted to a data receiving and processing platform (7) in real time through a power supply + data transmission line (6) for analysis and processing.

2. A method for performing binocular full-angle laser three-dimensional scanning inside a narrow pipe using the binocular full-angle laser three-dimensional scanning device inside a narrow pipe as claimed in claim 1, characterized in that: The method comprises the following steps: (1) Calibrate the binocular camera (2), which includes a left camera and a right camera, and obtain the internal parameters K of the left camera. l , Distortion coefficient of the left camera l , the internal parameters K of the right camera r , Distortion coefficient of the right camera r , the translation matrix T between the binocular cameras rl , rotation distance R rl , where the baseline distance between the binocular cameras is Baseline; in, Indicates the x-axis focal length of the left camera, Indicates the y-axis focal length of the left camera. Indicates the x-axis coordinate of the principal point in the left camera coordinate system. Indicates the y-axis coordinate of the principal point in the left camera coordinate system. Indicates the x-axis focal length of the right camera, Indicates the y-axis focus of the right camera, Indicates the x-axis coordinate of the principal point in the right camera coordinate system. Represents the y-axis coordinate of the principal point in the right camera coordinate system; (2) Use a binocular full-angle laser 3D scanning device to move at a constant speed inside the pipeline, and the movement speed is recorded as move v , using a binocular camera to collect binocular ring laser image sequences Images k = {I L (k),I R (k)}, I L (k) indicates the original image frame captured by the left camera, I R (k) indicates the original image frame captured by the right camera, k indicates the kth frame image, and the frame rate of the binocular camera is recorded as fps cam , the height average of the left and right images is H img , the width is W img The image sequence Images of the binocular ring laser is processed by using the internal parameters of the left camera, the internal parameters of the right camera, the translation matrix and the rotation matrix between the binocular cameras. k = {I L (k),I R (k)} is corrected and calibrated to obtain a new binocular image sequence ImagesC k = {Ic L (k),Ic R (k)},Ic L (k) represents the left eye image after correction and calibration, Ic R (k) represents the right eye image after right eye correction and calibration; (3) Using the laser stripe segmentation method based on GuideNet, the new binocular image sequence ImagesC obtained in step (2) is k = {Ic L (k),Ic R (k)}, the annular laser stripe area is quickly segmented to obtain the left annular laser stripe segmentation map Seg l And the right eye ring laser stripe segmentation diagram Seg r ; (4) Using the binocular ring laser centerline extraction method based on the Steger algorithm, the center lines of the ring laser stripe segmentation map of the left eye image and the ring laser stripe segmentation map of the right eye are extracted respectively, and the laser stripe center point set in the left eye image is obtained respectively. The center point set of the laser stripes in the right image Among them, Mc l ,Mc r Respectively represent the total number of center points detected in the left image and the right image; represents the image coordinates of the center point of the i-th left eye laser, Represents the image coordinates of the jth right eye laser center point; (5) Carry out 3D reconstruction of the laser center point in the middle section of the ring: Screen the left-eye ring laser stripe segmentation map Seg l And the right eye ring laser stripe segmentation diagram Seg r The center point of the laser stripe in the middle section of the ring is obtained by The center point set of the laser stripes located in the middle of the ring in the right image Among them, Mc l ,Mcc r represents the total number of laser stripe center points located in the middle section of the ring in the left image and the right image respectively, and uses the limit constraint principle and color similarity to calculate the set of laser stripe center points PCC located in the middle section of the ring in the left image. l and the center point set PCC of the laser stripes located in the middle section of the ring in the right image r Perform matching and determine the best matching point pair set M match Represents the total number of best matching points; calculates the points in the pair located in Seg according to the best matching point pair set MatchP l Pixels in The three-dimensional coordinates of Among them, the i mp Points The three-dimensional coordinates of Represent the three-dimensional coordinates of the x-axis, y-axis, and z-axis respectively; (6) Fitting the transient laser plane equation: Use the general formula A P X+B P Y+C P Z+D P = 0 represents the transient laser light plane equation at the current moment, where X, Y, and Z represent the coordinate variables in three-dimensional space. The simplex method (Nelder-Mead Algorithm) of unconstrained optimization is used, combined with The parameters {A P ,B P ,X P ,D P }, and obtain the transient laser light plane equation at the current moment; (7) 3D reconstruction of the full-angle annular laser ring based on transient laser plane fitting: Using the plane equation parameters, the transient laser light plane equation and the internal parameters of the left-eye camera are established to achieve the center points of all the annular lasers in the left-eye image of the kth frame. 3D reconstruction of (8) According to the moving distance, the circular laser scanning data acquired at each moment is spliced ​​to achieve the reconstruction of the three-dimensional point cloud data inside the pipeline Where Num is the total number of frames of the captured image.

3. The binocular full-angle laser three-dimensional scanning method for the interior of a narrow pipe according to claim 2 is characterized in that: The specific implementation steps of the laser stripe segmentation method based on GuideNet in step (3) are as follows: The left eye image Ic after correction and calibration L Take (k) as an example, (3-1) Preprocess the input image: Convert the original laser stripe image into a grayscale image I gray , and normalized I norm Represents the grayscale image I gray The normalized image is scaled to the standard size H×W:I input = resize(I norm , (H, W)), I input Represents the input image of the laser stripe segmentation method based on GuideNet, resize(I norm , (H,W)) means I norm The resizing operation is to scale the standard size H×W; (3-2) Will I input Input the initial convolution layer to extract the initial features; (a) For the input image I input Use three-branch parallel convolution, normalize them separately and then add them for feature fusion, and use ReLU activation function for activation to get the fusion feature F of the first stage. merge1 : F merg =ReLU(BN(Conv 3×3,S2 (I input ))+BN(Conv 3×1,s2 (I input ))+BN(Conv 1×3,s2 (I input )))(2) Among them, BN(·) represents normalization processing, ReLU(·) represents the ReLU activation function, Conv 3×3,S2 (·) indicates that a 3×3 convolution kernel with a stride of 2 is used for convolution processing. 3×1,S2 (·) indicates that the convolution kernel of 3×1 and stride 2 is used for convolution processing. 1×3,S2 (·) indicates convolution with a 1×3 kernel and a stride of 2; (b) Using 3×3 depth convolution DWConv with a stride of 2 3×3,S2 (·) and a 1×1 convolution kernel Conv with a stride of 2 1×1,S2 (·) The fusion features F of the first stage are respectively merge1 Convolution is performed, and the results after convolution are added and fused. Then, a 1×1 convolution kernel with a step size of 2 is used for dimensionality reduction. Finally, the ReLU activation function is used for activation to obtain the initial feature map F conv : F conv =ReLU(Conv 1×1,s2 (DWConv 3×3,s2 (F merge1 )+Conv 1×1,S2 (F merge1 ))) (3) (3-2) The preliminary feature map F conv Input a dual-branch encoder to extract high-level semantic features and high-resolution spatial information respectively; (a) The preliminary feature map F conv Input semantic feature extraction branch, and use 4 cascaded Transformer feature block extraction blocks to obtain four levels of high-level semantic feature maps Among them, MHSA(·) represents multi-head self-attention processing, FFN(·) represents feed-forward neural network processing; (b) F conv and four levels of high-level semantic feature maps The input spatial feature extraction branch is processed. The spatial feature extraction branch consists of four spatial feature extraction blocks and a guidance module cross-connected in cascade. sem ={1,2,3,4} spatial feature extraction blocks and guidance modules are combined as an example: Assume that the input of the spatial feature extraction block is and The output is Then with each layer of high-level semantic feature map The common input guidance module is used for feature fusion to obtain the fused guidance feature. First, calculate the depth-wise separable convolutional feature map Among them, DSC(·) represents the depth-wise separable convolution operation, DWConv 3×3,S1 (·) represents a 3×3 depthwise convolution with a stride of 1, PWConv 3×3,S1 (·) indicates point-wise convolution processing of 3×3 with a stride of 1; right After normalization and ReLU activation, we get: Use 3×3 convolution Conv with a step size of 1 respectively 3×1,S1 (·), 1×3, stride 1 convolution Conv 1×3,S1 (·), 3×1, stride 1 convolution Conv 3×1,S1 (·)right Perform convolution processing, normalize and add them separately, and get the spatial feature map after ReLU activation (c) Using the guidance module, the semantic feature maps of each layer are fused And the spatial feature map output by the spatial feature extraction block Get fusion guidance features First, calculate the semantic feature map and spatial feature map Similar pictures of Among them, σ(·) represents the Sigmoid activation function, UP(·) represents the upsampling of the semantic feature map to make it consistent with the shape of the spatial feature map, and ⊙ represents element-by-element multiplication; Based on similarity graph Calculate fusion feature map Further calculate the topological attention gate value Among them, IN(·) represents instance normalization, DSConv(·) represents dynamic convolution based on topological geometric constraints, The similarity between semantic features and spatial features guides weighted fusion features; Using topological attention gate value And fusion feature map Multiply pixel by pixel to get the fusion guidance feature (3-3) Fusion guide feature maps of different layers Input the multi-level feature fusion decoder, fuse the multi-level features, and gradually restore them to the original resolution to obtain the final laser stripe segmentation result: (a) First, Perform channel expansion to obtain the extended fusion guided feature map Calculate the attention weight of the i-th layer Through layer-by-layer weighted summation, multi-level features are fused: in, represents the highest-level feature map, and W4 represents the weight value of the highest-level feature map; (b) Setting the threshold τ seg , and obtain the final laser stripe binary segmentation result Seg of the left image l : (3-4) Input correction and right eye image Ic after correction processing R (k) Using the method of steps (3-1)-(3-3), the final laser stripe binary segmentation result Seg of the right eye image is obtained r .

4. The binocular full-angle laser three-dimensional scanning device and method for the interior of a narrow pipe according to claim 3 is characterized in that: The specific implementation steps of carrying out the three-dimensional reconstruction of the laser center point in the middle section of the ring in step (5) are as follows: (5-1) Assume that the ring laser stripe segmentation diagram Seg of the left image l Get the center point set of the laser stripes in the left image The first line that appears is marked as start cent , the last line is end cent , that is, the center point set of the laser stripes in the left image The x-axis coordinate range appears to be [start cent ,end cent ], select the row range that falls within [start cent +Thre cent ,end cent -Thre cent ] is taken as the center point set of the laser stripes located in the middle section of the ring in the left image, where: Similarly, select the right image where the row mark falls on [start cent +Thre cent ,end cent -Thre cent ] is taken as the center point set of the laser stripes in the middle section of the ring in the right image; after screening, the center point set in the middle section of the ring in the left image is recorded as The set of laser stripe center points located in the middle of the ring in the right image is recorded as Among them, Mc l ,Mcc r Respectively represent the total number of laser stripe center points located in the middle section of the ring in the left image and the right image; (5-2) For each laser center point in the middle section of the left image ring Calculate its color features Among them, round(·) means rounding off. The right eye candidate points are selected according to the epipolar constraint. The center point set in the middle section of the ring in the right eye image is In the example, find the row coordinates satisfying the epipolar constraint and the column coordinates in The points on the left are added to the matching candidate point set Cand: Among them, δ is the row coordinate constraint threshold. If there is no matching candidate point, the match is abandoned. Assume that we find J c indivual The matching candidate points of the set of matching candidate points Cand are calculated. c Color characteristics of each point: calculate The color feature difference between the candidate matching point And find the index corresponding to the minimum distance get The matching point Cand(j * ); (5-3) Calculate the matching point pairs of each laser center point in the middle section of the left image ring in turn to obtain the best matching point pair set M match Represents the total number of best matching point pairs; (5-4) The matching point pair MatchP located in the middle section of the ring is used as a reliable matching point pair for fitting the laser light plane. Combined with the baseline distance and the parallax of the left camera, the three-dimensional reconstruction MatchP of this part of the point cloud is realized. 3D , assuming i mp =1,2,3,…,M match , and Respectively and The column coordinates of and Respectively and The row coordinates of Then the three-dimensional coordinates of the matching points in the middle section of the left image ring can be calculated 5. The binocular full-angle laser three-dimensional scanning device and method for the interior of a narrow pipe according to claim 4, characterized in that: The specific implementation steps of the full-angle annular laser ring 3D reconstruction based on transient laser plane fitting in step (7) are as follows: (7-1) The center points of the laser stripes in the left image are collected The image coordinates are converted to normalized camera coordinates X n : in, and Respectively represent The x-axis coordinate and y-axis coordinate of a point in the normalized camera coordinate system, K l Represents the internal parameters of the left camera; (7-2) Use the transient laser light plane equation to calculate the three-dimensional coordinates of the center point of the ring laser: Define the light plane parameter vector A = [A P B P ], define the X and Y coordinates in normalized camera coordinates: in is the normalized camera coordinate of the pixel point, and the depth value Z(k) of the center point of all ring lasers in the k-th frame image is calculated: Calculate the X-axis and Y-axis three-dimensional coordinates XY(k) of the center points of all ring lasers in the k-th frame image: Among them, ⊙ represents element-by-element multiplication; (7-3) Then the final three-dimensional coordinates P of all the center points of the ring laser in the k-th frame image are 3D (k): Among them, fps cam Indicates the frame rate of the binocular camera, move v It indicates the movement speed of the binocular full-angle laser 3D scanning device moving at a uniform speed inside the pipe.

Images