A large cabin segment pose deviation measurement method based on monocular vision

Abstract

This invention provides a method for measuring the pose deviation of large-scale modules based on monocular vision, comprising: attaching measurement points to the sides of the mobile and fixed-end modules; placing tooling with end-face measurement points at the positioning pins and positioning holes on the docking end faces; acquiring the three-dimensional point coordinates of the sides and end-face measurement points of the mobile and fixed ends in the world coordinate system; transforming the origin of the coordinate system of the two sets of points obtained from measuring the mobile and fixed ends to the center of the end face of the module, and saving the transformed three-dimensional point coordinates and calibration matrix to the system database, with a one-to-one correspondence between the three-dimensional point coordinates and the module number; after the docking module arrives at the docking station, acquiring side images of the mobile and fixed ends, identifying the side measurement points of the fixed and mobile ends, and calculating the relative pose deviation between the mobile and fixed ends by combining the three-dimensional spatial coordinates of the side measurement points. This invention offers flexible target placement, fewer coordinate transformation steps, high measurement accuracy, and is simple and reliable.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of high-precision pose measurement, and particularly relates to a large cabin segment pose deviation measurement method based on monocular vision. BACKGROUND

[0002] In the field of aviation and aerospace manufacturing, the docking of large shell cabin segments has long relied on manual operation. Such large shell components are large in size and require high docking precision. Two or three people are needed to observe the cabin segment alignment from different positions, and it is often necessary to perform climbing operations, which poses certain safety hazards. Moreover, the adjustment amount cannot be intuitively quantified and is extremely dependent on the operating experience of the observation personnel. In this background, high-precision measurement requirements are proposed for the cabin segment docking production line to replace manual operation. High-precision digital measurement can be coordinated with an electric pose adjustment mechanism to realize the automation of the cabin segment measurement and docking process without the need for worker participation.

[0003] Patent No. CN105015800A discloses a method of measuring by using a laser tracker. A laser tracker target ball is used to measure the docking end face, and a T-Probe is used to track the cabin segment position to guide the docking process. The use of the laser tracker still requires high requirements for the on-site operators, who need to be specially trained. Moreover, this scheme uses a laser tracker, a T-Probe, and multiple target balls, and the overall scheme has a high cost, which is not conducive to widespread use.

[0004] Patent No. CN105910535A discloses a measurement method using a single camera and two cameras in combination. The two cameras are used to calibrate the cabin segments to be docked in an offline state. Two monocular cameras are used at the docking station to measure the fixed end and the moving end cabin segments, respectively. This scheme moves the calibration process to outside the measurement station and does not require manual participation during the docking process. However, due to the machining tolerance and surface shape error of the cabin segments, the use of an arc fixture to extract the pose information of the pin and hole and print it on the side surface of the cabin segment inevitably leads to poor adhesion of the arc fixture to the cabin segment, blurred edges of the printed feature points, poor roundness, and deviation from the actual positions of the pin and hole. The use of two cameras to measure the cabin segments separately also introduces conversion errors.

[0005] Patent No. CN105081719A discloses a method of using a single camera to guide the docking of cabin segments. A handheld mobile target is used to calibrate the relative position relationship of the docking cabin segments, and a fixed target is used to track the position of the moving end cabin segment. This method uses a single camera to measure the end face mobile target and the side surface fixed target. To ensure measurement accuracy, the mobile target and the fixed target are large in size. In the context of large cabin segment docking, the mobile target is inconvenient to operate, and the fixed target is difficult to install. SUMMARY

[0006] The present application aims to provide a large cabin segment pose deviation measurement method based on monocular vision, which is flexible in target arrangement position, has few coordinate conversion links, high measurement accuracy, simple and reliable.

[0007] The technical solution for achieving the object of the present application is:

[0008] A large cabin segment pose deviation measurement method based on monocular vision, comprising the steps of:

[0009] 1) Paste measurement points on the side surfaces of the mobile end and fixed end cabin segments, and place a tool with end surface measurement points at the positioning pins and positioning holes of the end surface;

[0010] 2) Use a photogrammetry device or a three-dimensional scanner to respectively obtain the three-dimensional point coordinates of the mobile end and fixed end measurement points in the world coordinate system;

[0011] 3) Convert the coordinate origins of the two sets of points obtained by measuring the mobile end and the fixed end to the cabin segment end surface centers, respectively, and save the converted three-dimensional point coordinates to the system database, and the three-dimensional point coordinates have a one-to-one correspondence with the cabin segment;

[0012] 4) Simultaneously obtain the images of the mobile end and fixed end side surface measurement points by using a single camera, and solve the relative pose deviation between the mobile end and the fixed end:

[0013] The measurement points pasted on the fixed end and the mobile end each have a specific arrangement order, so that the measurement system can distinguish the two parts of the target in the image, and the relative position and attitude of the camera relative to the two measurement points are obtained through PnP pose estimation of the recognized measurement points, and the relative pose deviation between the two cabin segments can be obtained through matrix transformation.

[0014] Compared with the prior art, the present application has the following beneficial effects:

[0015] (1) The side surface measurement points and the external calibration method are adopted, and there is no accurate requirement for the pasting position when pasting, and the pasting position can be removed after the docking is completed, so that no residue is left on the cabin surface, the three-dimensional point data after calibration is bound with the cabin information, saved to the database, and can be called at any time and used for docking;

[0016] (2) The calibration process uses a photogrammetry or three-dimensional scanning device to calibrate, and the calibration can be completed by hand, the accuracy can reach 0.03mm, the calibration cabin shape and size are not limited, the cabin can be calibrated in any factory building or at any position, the calibration process is separated from the docking process, the calibration file is called during docking, and the docking process is simplified;

[0017] (3) The single camera measurement method is used, the camera only measures the measurement point area, the measurement accuracy is high, compared with the binocular camera measurement method, the single camera occupies a small volume, is easy to install to the AGV, the mechanical arm and other narrow installation positions, and the field of view is not easy to be blocked;

[0018] (4) Compared with manual observation, 2-3 operators can be saved, manual observation and deviation judgment take about 30s, single camera measurement takes about 1.5s, and combined with the automatic posture adjusting mechanism, the docking time can be shortened to 3 minutes. BRIEF DESCRIPTION OF DRAWINGS

[0019] Fig. 1 It is a schematic diagram of the arrangement position of the measurement point of the cabin section.

[0020] Fig. 2 It is a schematic diagram of the installation of the measurement point of the end face of the cabin section.

[0021] Fig. 3 It is a schematic diagram of the calibration of the measurement point of the side face and the end face of the cabin section using a handheld three-dimensional scanner (taking the fixed end as an example).

[0022] Fig. 4 It is a schematic diagram of the single camera measurement of the relative pose deviation of the fixed end and the moving end of the cabin section. DETAILED DESCRIPTION

[0023] Embodiment 1

[0024] The embodiment provides a large cabin section pose deviation measurement method based on monocular vision, which specifically comprises the following steps:

[0025] 1. Paste the measurement point on the side face of the cabin section, the paste position is located on the side face of the cabin section, close to the position of the docking face, the paste area of the measurement point is 100mm*100mm, the measurement points of the fixed end and the moving end of the docking cabin section are located on the two sides of the docking face, and the circumferential positions are the same. The two measurement points are a fixed end side measurement point and a moving end side measurement point, and the two measurement points are composed of circular point type measurement points. The side measurement points are arranged according to a specific rule, and the side measurement points of the moving end and the fixed end are arranged in different ways, so that the two can be distinguished during measurement.

[0026] 2. Install the measurement point tool on the positioning pin and the positioning hole of the end face of the docking face, the measurement point tool has been inspected, and the center of the measurement point is located on the axis of the measurement point tool.

[0027] 3. Using a handheld three-dimensional scanner or photogrammetry equipment and other equipment capable of obtaining the three-dimensional spatial coordinates of the measurement points on the side and end surfaces, the relative positions of the measurement points on the side surface of the cabin section and the measurement points on the end surface of the cabin section are calibrated, the side surface measurement points and the end surface measurement points of the fixed end are saved in the same coordinate system, the side surface measurement points and the end surface measurement points of the moving end are saved in the same coordinate system, and the two coordinate systems are established at the center of the butt joint end surface, and when the two butt joint end surfaces are attached, the two coordinate systems coincide;

[0028] 4. The calibration information is bound with the cabin section number and saved in the database, and when the butt joint task starts, the corresponding calibration information is called from the database according to the number of the cabin section to be butt jointed;

[0029] 5. When the cabin section to be butt jointed reaches the butt joint station, a monocular camera obtains a picture of the side surface of the cabin section, identifies the side surface measurement points of the fixed end and the moving end according to the arrangement order of the side surface measurement points, and combines the three-dimensional spatial coordinates of the side surface measurement points in the center coordinate system to respectively calculate the relative positions and angle relationships between the side surface measurement points of the fixed end and the moving end and the camera, which are expressed in the form of homogeneous coordinates;

[0030] 6. The pose adjustment mechanism of the moving end cabin section moves in a certain direction, the camera measures the position of the moving end cabin section during the movement, records the movement direction as the movement direction of the pose adjustment mechanism, the pose adjustment mechanism moves in another direction, records the movement direction, establishes a pose adjustment mechanism coordinate system through the two directions, and the coordinate system is expressed in the form of homogeneous coordinates;

[0031] 7. The positions and angles of the fixed end and the moving end cabin section in the camera measurement coordinate system are converted to the center coordinate system through homogeneous coordinate conversion, and then the center coordinate system is converted to the pose adjustment mechanism coordinate system, and the positions and angles of the two are compared in the pose adjustment mechanism coordinate system.

[0032] Embodiment

[0033] In combination Figs. 1-4 A large cabin section pose deviation measurement method based on monocular vision, comprising the steps of:

[0034] 1. Measurement points 2 are pasted on the side surface of the fixed end cabin section 1, the pasting position is located on the side surface of the cabin section, close to the butt joint surface, the measurement point pasting area is 100mm*100mm, serving as the fixed end cabin section side surface measurement point 2, a work tool with measurement points is placed in the positioning hole of the cabin butt joint end surface, serving as the end surface measurement point 3 of the fixed end cabin section, the center of the measurement point 3 and the work tool axis have been calibrated in advance, the position error of the measurement point 3 is less than 0.01mm, and the gap between the work tool and the cabin positioning hole is 0.02mm;

[0035] 2. The measurement points of the moving end cabin section 6 are arranged in the same way as the fixed end, and the moving end side surface measurement points 5 and the end surface measurement points 4 are arranged.

[0036] The measurement points on the module section are calibrated. Taking the fixed-end module section as an example, a handheld 3D scanner 7 is used to calibrate the relative positions between the measurement points on the side of the module section and the measurement points on the end face of the module section, obtaining the coordinate values ​​of all measurement points in the world coordinate system P = {a1, a2, a3, a4, p1, p2, p3, p4, p5, ..., p n}, where a1-a4 are the coordinates of the end face measurement points, p1, p2, p3...p n For the coordinates of the side measurement points, during calibration, the coordinates of the side measurement points and the end face measurement points are in the same coordinate system. The origin of the coordinate system for the measurement point P is transferred to the center of the end face of the compartment. The midpoint of the line connecting a1a3 and a2a4 is taken as the origin. The direction to a3 is the positive X-axis direction, and the direction to a2 is the positive γ-axis direction. The cross product of these two directions yields the Z-axis direction vector. From this, the center coordinate system A can be established. Fo =[R Fo t Fo ], R Fo The rotation matrix of the fixed-end coordinate system origin with respect to the center of the end face of the compartment is t, which is a 4x3 matrix in this embodiment. Fo Let P be the translation vector of the origin of the fixed-end coordinate system with respect to the center of the end face of the compartment, used to represent the coordinate system. In this embodiment, it is a 4x1 translation vector. Then, the coordinates P of the fixed-end measurement point in the center coordinate system are represented as P. Fc =(Pt Fo )·R Fo Using the same method, the coordinates P of the measurement point of the mobile terminal section in the coordinate system of the mobile terminal section's center can be obtained. Mc =(Pt o )·R o , will P Fc P Mc The calibration matrix calculation result, as the calibration result, is bound to the compartment number and saved to the database. o Let t be the rotation matrix of the origin of the mobile coordinate system with respect to the center of the end face of the compartment. o Let be the translation vector of the origin of the mobile coordinate system with respect to the center of the end face of the compartment;

[0037] 3. After docking begins, camera 8 captures images of the target position on the side of the docking module, extracts the measurement points from the images, and for the four collinear measurement points p1, p2, p3, and p4, their projection points in the image are as follows: Based on the invariance of cross ratio, we have

[0038]

[0039] Based on the different cross ratios of the calibrated measurement points, the fixed end side measurement points and the moving end side measurement points of the compartment can be identified and distinguished.

[0040] 4, three-dimensional space coordinates p i = (x i , y i , z i ) and its projection coordinates in the image There is a conversion relationship as follows:

[0041]

[0042] Wherein is the camera intrinsic matrix, which can be obtained by camera intrinsic calibration, R is the rotation matrix of the camera relative to the measuring point, T is the translation of the camera relative to the measuring point, R and T are obtained by EPnP algorithm, the relationship between the camera coordinate system and the measuring point coordinate system (center coordinate system) satisfies A c = P·R+T, P is the center coordinate system, A c is the camera coordinate system, that is, the homogeneous matrix, and the embodiment is a 4x4 homogeneous matrix;

[0043] 5, record the initial position measurement value P1 of the cabin section (the position of the cabin section coordinate system in the measuring coordinate system, the cabin section coordinate system takes the center of the cabin end face as the origin, and the measuring coordinate system refers to the world coordinate system), the cabin section moves 200mm along the axial direction from the initial position, records the current cabin section position P2, the cabin section returns to the initial position P1 and moves upward by 50mm, records the current position P3, takes the P1 coordinate value as the origin, the P1P2 direction is the positive direction of X axis, the P1P3 direction is the positive direction of Y axis, and the Z axis direction vector is obtained by the cross product of the two, so that the pose adjustment mechanism coordinate system can be established The aforementioned X axis, Y axis and Z axis and three direction vectors, divided by their own module length, obtain their standard vectors respectively denoted as The P1 coordinate value is

[0044] 6, convert the center coordinate system of the fixed end and the moving end of the cabin section into the pose adjustment mechanism coordinate system, and the converted coordinate systems of the two fixed ends and the moving end are respectively denoted as Through translation and rotation of to coincide with , that is Solve the equation set to obtain the deviation value of the two parts of the cabin section in the pose adjustment mechanism coordinate system, wherein R is the rotation matrix of the center coordinate system relative to the pose adjustment mechanism coordinate system, and T is the translation of the center coordinate system relative to the pose adjustment mechanism coordinate system.

[0045] Compared with manual observation, the cabin segment pose deviation measurement method based on monocular vision can save 2-3 operators, the manual observation and judgment deviation takes about 30s, the single measurement time of the camera is about 1.5s, and combined with the automatic pose adjusting mechanism, the docking time can be shortened to 3min.

Claims

1. A monocular vision-based large cabin segment pose deviation measurement method, characterized in that, The method comprises the steps of: Step 1, pasting measurement points on the side surface of the mobile end and the fixed end cabin section, and placing a positioning pin and a positioning hole on the end surface of the joint end surface with a tool with end surface measurement points; Step 2, obtaining three-dimensional point coordinates of the side surface and end surface measurement points of the mobile end and the fixed end in the world coordinate system; Step 3, calibrating the side surface and end surface measurement points, i.e. converting the coordinate system origin points of the two sets of point sets obtained by measuring the mobile end and the fixed end to the center of the cabin end surface, and saving the converted three-dimensional point coordinates and the calibration matrix to the system database, and the three-dimensional point coordinates have a one-to-one correspondence with the cabin number; Step 4, after the joint cabin reaches the joint station, obtaining the side surface images of the mobile end and the fixed end, identifying the side surface measurement points of the fixed end and the mobile end, and combining the three-dimensional space coordinates of the side surface measurement points to solve the relative pose deviation between the mobile end and the fixed end; The step 3 comprises: calibrating the relative positions between the cabin side surface measurement points and the cabin end surface measurement points, the side surface measurement points and the end surface measurement points of the fixed end are saved in the same coordinate system, the side surface measurement points and the end surface measurement points of the mobile end are saved in the same coordinate system, and the two coordinate systems are established at the center of the joint end surface, and the two coordinate systems coincide when the two joint end surfaces are attached; The step 3 of converting the coordinate system origin points of the two sets of point sets obtained by measuring the mobile end and the fixed end to the center of the cabin end surface specifically comprises: Another fixed end side and end surface measurement point coordinate value Wherein End surface measurement point coordinate value, , , ... Side surface measurement point coordinate value, the side surface measurement point and the end surface measurement point coordinate value are in the same world coordinate system; Transfer the origin of the coordinate system of the measurement point P to the center of the circle on the end face of the fixed end section, and take... , The midpoint of the line is taken as the origin. The direction is the positive X-axis. The direction is the positive Y-axis; the cross product of the two directions yields the Z-axis direction vector. Establish a coordinate system centered on the circle. ,in For the rotation matrix of the origin of the fixed-end measurement coordinate system with respect to the center of the end face of the compartment, The translation vector of the origin of the coordinate system at the fixed end relative to the center of the end face of the compartment; Then the coordinate of the fixed end measuring point P in the center coordinate system is i.e. the calibration matrix; Repeating the method to obtain the calibration matrix of the mobile end cabin measurement points in the mobile end cabin center coordinate system; The step 4 specifically comprises: After the joint starts, the camera shoots the side surface target position of the joint cabin, obtains the picture, extracts four collinear measurement points in the picture, identifies and distinguishes the fixed end side surface measurement points and the mobile end side surface measurement points according to the different cross ratios of the calibrated measurement points; According to the three-dimensional space coordinates of the converted measurement points and the projection coordinates in the image, a homogeneous matrix for converting the camera coordinate system and the measurement point coordinate system is obtained; Establishing a pose adjustment mechanism coordinate system; According to the homogeneous matrix, the camera coordinate system is converted to the center coordinate system, and then the center coordinate system is converted to the pose adjustment mechanism coordinate system, and the center coordinate system and the pose adjustment mechanism coordinate system are respectively represented as , , by translating and rotating to coincide with , that is , the deviation values of the two-part cabin section in the pose adjustment mechanism coordinate system are obtained by solving, wherein R is the rotation matrix of the center coordinate system relative to the pose adjustment mechanism coordinate system, and T is the translation amount of the center coordinate system relative to the pose adjustment mechanism coordinate system.

2. The monocular vision-based large cabin segment pose deviation measurement method according to claim 1, characterized in that, The side surface measurement points are regularly arranged by circular point type measurement points, and the arrangement modes of the side surface measurement points of the mobile end and the fixed end are different, the circumferential positions of the pasted side surface measurement points are the same, and the pasted area is 100mm*100mm.

3. The monocular vision-based large cabin segment pose deviation measurement method according to claim 1, characterized in that, The end surface measurement points are four, two by two symmetric, and the gap between the end surface measurement point tool and the cabin positioning hole is 0.02mm.

4. The monocular vision-based large cabin segment pose deviation measurement method according to claim 1, characterized in that, The three-dimensional point coordinates in the world coordinate system in the step 2 are obtained by a handheld three-dimensional scanner or a photogrammetry device.

5. The monocular vision-based large cabin segment pose deviation measurement method according to claim 1, characterized in that, The side surface images of the mobile end and the fixed end in the step 4 are obtained by a monocular camera.

6. The monocular vision-based large cabin segment pose deviation measurement method according to claim 1, characterized in that, The homogeneous matrix for converting the camera coordinate system and the measurement point coordinate system comprises: Acquiring three-dimensional spatial coordinates and their conversion relationship with the projected coordinates in the image: and their conversion relationship with the projected coordinates in the image: ； wherein is a camera intrinsic matrix, obtained by camera intrinsic calibration, R is a rotation matrix of the camera relative to the measuring point, T is a translation of the camera relative to the measuring point, R and T are obtained by an EPnP algorithm, and a homogeneous matrix is determined as: , P is a center coordinate system.

7. The monocular vision-based large cabin segment pose deviation measurement method according to claim 1, characterized in that, The establishment of the pose adjustment mechanism coordinate system specifically comprises: Record the initial measurement position of the compartment. Move the compartment 200mm from its initial position along the axis and record its current position. The compartment returned to its initial position. Move upwards by 50mm and record the current position. ,Pick The coordinate values ​​are used as the origin. The direction is the positive X-axis. The direction is the positive Y-axis direction, and the cross product of the two gives the Z-axis direction vector; The standard vectors of X-axis, Y-axis and Z-axis are , , , The coordinate values are The coordinate system of the pose adjustment mechanism is established .

Images