Duct piece assembly target position pose obtaining method based on unmanned aerial vehicle depth vision

Abstract

The invention discloses an unmanned aerial vehicle depth vision-based position and posture acquisition method for a duct piece assembly target position. The method comprises the following steps of scanning related duct pieces associated with a to-be-mounted position through a depth vision sensor mounted on an unmanned aerial vehicle; based on the local coordinate system of the unmanned aerial vehicle, obtaining all point cloud data in the scanning range of the depth vision sensor and extracting side lines; setting three points corresponding to the target position of the segment to be assembled, and determining the spatial position relationship between the three points and the end points of the three side lines on the installed segment; converting the cylindrical coordinates of the target position corresponding to the three points into rectangular coordinates; calculating a target attitude matrix of the target position of the segment to be assembled through the rectangular coordinates of the three points corresponding to the target position; calculating the coordinates of the target position of the segment to be assembled; and calculating a pose matrix of the target position of the segment to be assembled. The invention relates to the technical field of shield tunnel construction, and can solve the problem that in the prior art, the target splicing position of a to-be-spliced segment is difficult to accurately obtain in a limited space environment in a shield tunneling machine.

Description

Technical Field

[0001] The present invention relates to the technical field of shield tunnel construction, and in particular to a method for acquiring the position and posture of a target position of a segment assembly based on the depth vision of an unmanned aerial vehicle. Background Art

[0002] The development of shield technology has been completed since the 1980s. Although the coordination between the various systems and components of the shield machine has been basically completed, it still relies heavily on manual work experience in the interface between geotechnical equipment, equipment and equipment, and equipment and people. The assembly machine is a key component for positioning and installing prefabricated segments. The assembly workers adjust the six-degree-of-freedom motion of the segments by operating the handle buttons to make their posture meet the assembly requirements.

[0003] Due to the limited visibility, poor lighting conditions, complex working environment, and uneven operating experience of assembly workers in the tunnel, it is impossible to accurately obtain the target assembly position of the segments to be assembled, resulting in fluctuations in the quality of tunnel forming. Therefore, it is necessary to provide a method for obtaining the position of the target assembly position of the segments based on the depth vision of drones, which can solve the problem of the difficulty in accurately obtaining the target assembly position of the segments to be assembled in the limited space environment inside the shield machine in the prior art. Summary of the invention

[0004] The purpose of the present invention is to provide a method for acquiring the position and posture of the target assembly position of the segment based on the depth vision of an unmanned aerial vehicle, which can solve the problem in the prior art that it is difficult to accurately acquire the target assembly position of the segment to be assembled in the limited space environment inside a shield machine.

[0005] The present invention is achieved in that:

[0006] A method for acquiring the position and posture of a target position of a segment assembly based on the depth vision of an unmanned aerial vehicle comprises the following steps:

[0007] Step 1: Scan the relevant pipe segments associated with the location to be installed using the depth vision sensor installed on the drone;

[0008] Step 2: Based on the local coordinate system of the drone, obtain all point cloud data P within the scanning range of the depth vision sensor i (r i ,θ i ,h i ) and extract the edge lines of the installed segments;

[0009] Step 3: Set three points corresponding to the target position of the segment to be assembled, and determine the relative spatial position relationship between the three points and the three edge line endpoints of the installed segment;

[0010] Step 4: Convert the cylindrical coordinates of the three points corresponding to the target position into rectangular coordinates;

[0011] Step 5: Calculate the target posture matrix of the target position of the segment to be assembled through the rectangular coordinates of the three points corresponding to the target position;

[0012] Step 6: Calculate the coordinates of the target position of the segment to be assembled;

[0013] Step 7: Calculate the pose matrix of the target position of the segment to be assembled.

[0014] The step 1 comprises the following sub-steps:

[0015] Step 1.1: Install a depth vision sensor on the drone;

[0016] Step 1.2: Plan the motion path of the UAV so that when the segments to be assembled are in different postures within the assembly area, the scanning range of the depth vision sensor can cover the segments to be assembled and the associated areas of the installed segments adjacent to the segments to be assembled.

[0017] In step 2, the extracted edge lines of the installed segments include: the inner arc end surface line segment AB of the third segment in the same ring as the segment to be assembled, two inner arc annular circular curves AE and BF, and the inner arc annular circular curve CD of the first segment and the second segment in the previous ring.

[0018] The method for extracting the inner arc surface annular circular curve CD is as follows: the line segment BA is extended to a point with the same θ value in the upper edge circular curve of the first segment of the previous ring, and according to the known arc length C'D' of the inner arc surface of the segment to be assembled, after the inner arc surface annular circular curves of the first segment and the second segment of the previous ring are extracted, a circular curve equal to the arc length C'D' of the inner arc surface of the segment to be assembled is cut with C as the starting point, and the end point of the circular curve is point D.

[0019] In step 3, the edge endpoints on the installed segment include: point B of the inner arc end surface line segment AB, and points C and D of the inner arc torus circular curve CD. The cylindrical coordinates of points B, C, and D are B(r B ,θ B ,h B )、C(r C ,θ C ,h C ), D(r D ,θ D ,h D );

[0020] The three points corresponding to the target position of the segment to be assembled are manually set as point B', point C' and point D', wherein point B on the installed segment corresponds to point B' of the segment to be assembled, point C on the installed segment corresponds to point C' of the segment to be assembled, and point D on the installed segment corresponds to point D' of the segment to be assembled;

[0021] The cylindrical coordinates of points B', C', and D' in the local coordinate system of the drone are expressed as follows:

[0022]

[0023]

[0024] In step 4, the rectangular coordinates of the three points B', C', and D' corresponding to the target position are:

[0025] B′=(x B ,y B ,z B )=(r B1 cosθ B1 ,r B1 sinθ B1 ,h B1 )

[0026] C′=(x C ,y C ,z C )=(r C1 cosθ C1 ,r C1 sinθ C1 ,h C1 )

[0027] D′=(x D ,y D ,z D )=(r D1 cosθ D1 ,r D1 sinθ D1 ,h D1 ).

[0028] The step 5 comprises the following sub-steps:

[0029] Step 5.1: Based on the rectangular coordinates of points B', C', and D', calculate and vector, and make the normal vectors of the plane where points B', C', and D' are located It is expressed as:

[0030]

[0031] Step 5.2: The target posture matrix T1 of the target position of the segment to be assembled is expressed as:

[0032]

[0033] The step 6 comprises the following sub-steps:

[0034] Step 6.1: Obtain the rectangular coordinates of the midpoint G of the line HJ connecting hole No. 1 H and hole No. 2 J on the segment to be assembled;

[0035] Step 6.2: On the segment to be assembled, the lengths of the three points B', C', D' and the midpoint G are fixed and known. Let the length of B'G be m1, the length of C'G be m2, and the length of D'G be m3;

[0036] Step 6.3: Let the G coordinate be (x G ,y G , z G ), then it is only necessary to satisfy that the distances between B', C' and D' and point G are equal to m1, m2 and m3 respectively, B'G = m1; C'G = m2; D'G = m3;

[0037] Step 6.4: Calculate the coordinates of the midpoint G;

[0038] The coordinates of B', C' and D' are (x B′ ,y B′ ,z B′ ), (x C′ ,y C′ ,z C′ ) and (x D′ ,y D′ ,z D′ ),satisfy:

[0039]

[0040] Using the above three equations, we can solve for x G ,y G , z G , thus obtaining the midpoint G coordinate.

[0041] In step 7, according to the origin O and the midpoint G of the rectangular coordinate system, That is, the target position of the segment to be assembled; the target position of the segment to be assembled is combined with the target posture matrix T1 to obtain the pose matrix T2 of the target position:

[0042]

[0043] Compared with the prior art, the present invention has the following beneficial effects:

[0044] 1. The present invention adopts a drone equipped with a depth vision sensor to scan the area of ​​the to-be-assembled segment and its adjacent segments, and utilizes the conversion relationship between the drone's cylindrical coordinate system and the rectangular coordinate system to calculate the target posture matrix of the to-be-assembled segment and combine it with the target position of the to-be-assembled segment to obtain the posture matrix of the target position. This method does not rely on manual experience, and effectively improves the accuracy and efficiency of obtaining the posture matrix of the target position of the to-be-assembled segment, thereby solving the problem of difficulty in accurately obtaining the target assembly position of the to-be-assembled segment in the limited space environment inside the shield machine.

[0045] 2. The present invention adopts a drone equipped with a depth vision sensor to assist in acquiring the pose matrix of the target position of the segments to be assembled, thereby solving the problem that the naked eye cannot quickly and accurately acquire the pose matrix of the target position of the segments to be assembled when there is no visibility in the limited space inside the shield machine. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 It is an operation schematic diagram of the method for acquiring the position and posture of the target position of the segment assembly based on the depth vision of the unmanned aerial vehicle of the present invention;

[0047] Figure 2 It is a principle diagram of the spatial position relationship between point D and point D' in the method for acquiring the position and posture of the target position of segment assembly based on the depth vision of the unmanned aerial vehicle of the present invention;

[0048] Figure 3 It is a schematic diagram of step 6 in the method for acquiring the position and posture of the target position of the segment assembly based on the depth vision of the unmanned aerial vehicle of the present invention.

[0049] In the picture, 1 is a drone, 2 is a depth vision sensor, 3 is a pipe segment to be assembled, and 4 is an installed pipe segment. DETAILED DESCRIPTION

[0050] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments.

[0051] A method for acquiring the position and posture of a target position of a segment assembly based on the depth vision of an unmanned aerial vehicle comprises the following steps:

[0052] Please see attached Figure 1 , Step 1: Scan the relevant pipe segments associated with the location to be installed through the depth vision sensor 2 installed on the drone 1.

[0053] The step 1 comprises the following sub-steps:

[0054] Step 1.1: Install the depth vision sensor 2 on the drone 1.

[0055] Preferably, the depth vision sensor 2 can adopt a binocular vision sensor, such as the Hikvision robot binocular perception camera model MV-EB435i.

[0056] Step 1.2: Plan the motion path of the drone 1 so that when the segment 3 to be assembled is in different posture states in the assembly area, the scanning range of the depth vision sensor 2 can cover the segment 3 to be assembled and the associated area of ​​the installed segment 4 adjacent to the segment 3 to be assembled.

[0057] Preferably, the motion path planning of the drone 1 can be determined according to the actual construction environment to ensure that the scanning range of the depth vision sensor 2 meets the construction requirements. At present, the segment assembly usually adopts staggered assembly, so the installed segment 4 adjacent to the segment 3 to be assembled usually has three segments, as shown in the attached figure. Figure 1 As shown in FIG. 1 , the three installed segments 4 are the first segment K1 and the second segment K2 of the previous ring and the third segment K3 of the same ring. The scanning range of the depth vision sensor 2 on the drone 1 should cover the segment to be assembled 3 and the splicing edges of the segment to be assembled 3 and the first segment K1, the second segment K2 and the third segment K3, as shown in FIG. Figure 1 As shown in the box line.

[0058] Step 2: Based on the local coordinate system of UAV 1 (the local coordinate system is a cylindrical coordinate system), obtain all point cloud data P within the scanning range of the depth vision sensor 2 i (r i ,θ i ,h i ) and extract the edge line of the installed segment 4.

[0059] In the step 2, the extracted edge lines of the installed segments 4 include: the inner arc end surface line segment AB of the third segment K3 in the same ring as the segment 3 to be assembled, two inner arc annular circular curves AE and BF, and the inner arc annular circular curve CD of the first segment K1 and the second segment K2 in the previous ring.

[0060] The method for extracting the inner arc surface annular circular curve CD is: the line segment BA is extended to the point with the same θ value in the upper edge line circular curve of the first pipe segment K1 of the previous ring, according to the known arc length C'D' of the inner arc surface of the pipe segment 3 to be assembled, after the inner arc surface annular circular curves of the first pipe segment K1 and the second pipe segment K2 of the previous ring are extracted, C is used as the starting point to cut a circular curve equal to the arc length C'D' of the inner arc surface of the pipe segment 3 to be assembled, and the end point of the circular curve is point D.

[0061] Step 3: Set three points corresponding to the target position of the pipe segment 3 to be assembled, and determine the relative spatial position relationship between the three points and the three edge line endpoints on the installed pipe segment 4.

[0062] In step 3, the edge endpoints on the installed segment 4 include: point B of the inner arc end surface line segment AB, and points C and D of the inner arc torus circular curve CD. The cylindrical coordinates of points B, C, and D are B(rB ,θ B ,h B )、C(r C ,θ C ,h C ), D(r D ,θ D ,h D ).

[0063] The three points corresponding to the target position of the segment 3 to be assembled are manually set as point B', point C', and point D', wherein point B on the installed segment 4 corresponds to point B' of the segment 3 to be assembled, point C on the installed segment 4 corresponds to point C' of the segment 3 to be assembled, and point D on the installed segment 4 corresponds to point D' of the segment 3 to be assembled. The relative spatial position relationship between points B', C', and D' and points B, C, and D is determined.

[0064] Please see attached Figure 2 , take point D as an example to explain: Δr D is the vertical distance that should be maintained between D' and point D in the r-axis direction; Δl D Δh is the circumferential distance between D' and D in the direction of rotation at angle θ; D is the longitudinal distance that should be maintained between D' and point D in the h-axis direction.

[0065] Therefore, the cylindrical coordinates of points B', C', and D' in the local coordinate system of UAV 1 can be expressed as follows:

[0066]

[0067] Step 4: Convert the cylindrical coordinates of the three points B', C', and D' corresponding to the target position into rectangular coordinates:

[0068] B′=(x B ,y B ,z B )=(r B1 cosθ B1 ,r B1 sinθ B1 ,h B1 )

[0069] C′=(x C ,y C ,z C )=(r C1 cosθ C1 ,r C1 sinθ C1 ,h C1 )

[0070] D′=(x D ,yD ,z D )=(r D1 cosθ D1 ,r D1 sinθ D1 ,h D1 ).

[0071] The conversion of the coordinates of the cylindrical coordinate system and the rectangular coordinate system is carried out by conventional mathematical calculation methods, which will not be described in detail here.

[0072] Step 5: Calculate the target posture matrix of the target position of the segment 3 to be assembled through the rectangular coordinates of the three points corresponding to the target position.

[0073] The step 5 comprises the following sub-steps:

[0074] Step 5.1: Based on the rectangular coordinates of points B', C', and D', calculate and vector, and make the normal vectors of the plane where points B', C', and D' are located It is expressed as:

[0075]

[0076] Step 5.2: The target posture matrix T1 of the target position of the segment 3 to be assembled can be expressed as:

[0077]

[0078] Step 6: Calculate the coordinates of the target position of the segment 3 to be assembled.

[0079] Please see attached Figure 2 , the step 6 comprises the following sub-steps:

[0080] Step 6.1: Obtain the rectangular coordinates of the midpoint G of the line HJ connecting hole No. 1 H and hole No. 2 J on the pipe segment 3 to be assembled.

[0081] Step 6.2: On the segment 3 to be assembled, the lengths of the three points B', C', D' and the midpoint G are fixed and known. Let the length of B'G be m1, the length of C'G be m2, and the length of D'G be m3.

[0082] Step 6.3: Let the coordinates of the midpoint G be (x G ,y G , z G ), then it is only necessary to satisfy that the distances between B', C' and D' and point G are equal to m1, m2 and m3 respectively, B'G = m1; C'G = m2; D'G = m3.

[0083] Step 6.4: Calculate the coordinates of the midpoint G.

[0084] The coordinates of B', C' and D' are (x B′ ,y B′ ,z B′ ), (x C′ ,y C′ ,z C′ ) and (x D′ ,y D′ ,z D′ ),satisfy:

[0085]

[0086] Using the above three equations, we can solve for x G ,y G , z G , thus obtaining the midpoint G coordinate.

[0087] Step 7: Calculate the pose matrix of the target position of the segment 3 to be assembled (the pose matrix of the target position is a 4*4 pose matrix).

[0088] Specifically, according to the origin O and midpoint G of the rectangular coordinate system, we get That is, the target position of the segment 3 to be assembled. The target position of the segment 3 to be assembled is combined with the target posture matrix T1 to obtain the pose matrix T2 of the target position. The pose matrix T2 of the target position can be used to obtain the dual information of the posture and position of the segment 3 to be assembled, which is more convenient for the segment assembly construction.

[0089]

[0090] The above are only preferred embodiments of the present invention and are not intended to limit the protection scope of the present invention. Therefore, any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for acquiring the position and posture of a target position of segment assembly based on the depth vision of an unmanned aerial vehicle, characterized by: The following steps are involved: Step 1: Scanning relevant pipe segments associated with the location to be installed by using a depth vision sensor (2) installed on the drone (1); Step 2: Based on the local coordinate system of the drone (1), obtain all point cloud data P within the scanning range of the depth vision sensor (2) i (r i ,θ i ,h i ), and extract the edge line of the installed segment (4); Step 3: setting three points corresponding to the target position of the pipe segment (3) to be assembled, and determining the relative spatial position relationship between the three points and the three edge line endpoints of the installed pipe segment (4); Step 4: Convert the cylindrical coordinates of the three points corresponding to the target position into rectangular coordinates; Step 5: Calculate the target posture matrix of the target position of the segment to be assembled (3) through the rectangular coordinates of the three points corresponding to the target position; Step 6: Calculate the coordinates of the target position of the segment to be assembled (3); Step 7: Calculate the pose matrix of the target position of the segment to be assembled (3).

2. The method for acquiring the position and posture of the target position of segment assembly based on the depth vision of an unmanned aerial vehicle according to claim 1 is characterized by: The step 1 comprises the following sub-steps: ； Step 1.1: Install a depth vision sensor (2) on the drone (1); Step 1.2: The motion path of the drone (1) is planned so that when the segment to be assembled (3) is in different posture states in the assembly area, the scanning range of the depth vision sensor (2) can cover the segment to be assembled (3) and the associated area of ​​the installed segment (4) adjacent to the segment to be assembled (3).

3. The method for acquiring the position and posture of the target position of segment assembly based on the depth vision of an unmanned aerial vehicle according to claim 1 is characterized by: In the step 2, the extracted edge lines of the installed segments (4) include: the inner arc end surface line segment AB of the third segment K3 in the same ring as the segment to be assembled (3), two inner arc annular circular curves AE and BF, and the inner arc annular circular curve CD of the first segment K1 and the second segment K2 in the previous ring.

4. The method for acquiring the position and posture of the target position of segment assembly based on the depth vision of an unmanned aerial vehicle according to claim 3 is characterized by: The method for extracting the inner arc surface annular circular curve CD is as follows: the line segment BA is extended to a point having the same θ value in the upper edge line circular curve of the first pipe segment K1 of the previous ring; according to the known arc length C'D' of the inner arc surface of the pipe segment (3) to be assembled, after the inner arc surface annular circular curves of the first pipe segment K1 and the second pipe segment K2 of the previous ring are extracted, a circular curve equal to the arc length C'D' of the inner arc surface of the pipe segment (3) to be assembled is cut with C as the starting point, and the end point of the circular curve is point D.

5. The method for acquiring the position and posture of the target position of segment assembly based on the depth vision of an unmanned aerial vehicle according to claim 1 is characterized by: In the step 3, the edge endpoints on the installed segment (4) include: point B of the inner arc end surface line segment AB, and points C and D of the inner arc torus circular curve CD. The cylindrical coordinates of the three points B, C and D are B(r B ,θ B ,h B )、C(r C ,θ C ,h C ), D(r D ,θ D ,h D ); The three points corresponding to the target position of the pipe segment (3) to be assembled are manually set as point B', point C' and point D', wherein point B on the installed pipe segment (4) corresponds to point B' of the pipe segment (3) to be assembled, point C on the installed pipe segment (4) corresponds to point C' of the pipe segment (3) to be assembled, and point D on the installed pipe segment (4) corresponds to point D' of the pipe segment (3) to be assembled; The cylindrical coordinates of points B', C', and D' in the local coordinate system of the drone (1) are expressed as follows:

6. The method for acquiring the position and posture of the target position of segment assembly based on the depth vision of an unmanned aerial vehicle according to claim 1 is characterized by: In step 4, the rectangular coordinates of the three points B', C', and D' corresponding to the target position are: B′=(x B ,y B ,z B )=(r B1 cosθ B1 ,r B1 sinθ B1 ,h B1 ) C′=(x C ,y C ,z C )=(r C1 cosθ C1 ,r C1 sinθ C1 ,h C1 ) D′=(x D ,y D ,z D )=(r D1 cosθ D1 ,r D1 sinθ D1 ,h D1 )。 7. The method for acquiring the position and posture of the target position of segment assembly based on the depth vision of an unmanned aerial vehicle according to claim 1 is characterized by: The step 5 comprises the following sub-steps: Step 5.1: Based on the rectangular coordinates of points B', C', and D', calculate and vector, and make the normal vectors of the plane where points B', C', and D' are located It is expressed as: Step 5.2: The target posture matrix T1 of the target position of the segment to be assembled (3) is expressed as:

8. The method for acquiring the position and posture of the target position of segment assembly based on the depth vision of an unmanned aerial vehicle according to claim 1 is characterized by: The step 6 comprises the following sub-steps: Step 6.1: Obtain the rectangular coordinates of the midpoint G of the line HJ connecting the No. 1 hole H and the No. 2 hole J on the segment to be assembled (3); Step 6.2: On the segment to be assembled (3), the lengths of the three points B', C', D' and the midpoint G are fixed and known. Let the length of B'G be m1, the length of C'G be m2, and the length of D'G be m3; Step 6.3: Let the G coordinate be (x G ,y G , z G ), then it is only necessary to satisfy that the distances between B', C' and D' and point G are equal to m1, m2 and m3 respectively, B'G = m1; C'G = m2; D'G = m3; Step 6.4: Calculate the coordinates of the midpoint G; The coordinates of B', C' and D' are (x B′ ,y B′ ,z B′ ), (x C′ ,y C′ ,z C′ ) and (x D′ ,y D′ ,z D′ ),satisfy: The above three equations can be used to solve for x G ,y G , z G , thus obtaining the midpoint G coordinate.

9. The method for acquiring the position and posture of the target position of segment assembly based on the depth vision of an unmanned aerial vehicle according to claim 1, characterized in that: In step 7, according to the origin O and the midpoint G of the rectangular coordinate system, That is, the target position of the segment to be assembled (3); the target position of the segment to be assembled (3) is combined with the target posture matrix T1 to obtain the posture matrix T2 of the target position:

Images