Airplane deformation displacement measurement method based on monocular camera and laser range finder

Abstract

The invention belongs to the technical field of measurement, and relates to an aircraft deformation displacement measurement method based on a monocular camera and a laser range finder, which comprises the following steps: 1) building a mobile robot measurement system; 2) completing position calibration of the monocular camera and the laser range finder; 3) giving a transformation matrix TR, M between a robot coordinate system and a map coordinate system; 4) determining a transformation matrix TP, R between the robot coordinate system and the aircraft coordinate system; 5) obtaining a transformation matrix TP, M between the aircraft coordinate system and the map coordinate system based on TR, M and TP, R; 6) obtaining the coordinate Si of the optimal measurement point in the aircraft coordinate system, and obtaining the coordinate Mn of the optimal measurement point in the map coordinate system through coordinate transformation based on the TP and M; and 7) the measurement robot performs multi-point path planning according to the coordinate Mn of the optimal measurement point in the map coordinate system, and sequentially moves to the position of the optimal measurement point to perform aircraft deformation measurement. The measuring efficiency is high, the practicability is high, and automatic and intelligent measurement of aircraft deformation can be achieved.

Description

Technical Field

[0001] The present invention belongs to the field of measurement technology and relates to a method for measuring aircraft deformation and station shifting, and in particular to a method for measuring aircraft deformation and station shifting based on a monocular camera and a laser rangefinder. Background Art

[0002] This background information merely provides background information related to the present invention and does not necessarily constitute prior art.

[0003] During flight missions and daily operations, aircraft are subject to external environmental factors such as pressure and temperature, causing deformation of the fuselage structure. This deformation can lead to stress concentrations in certain areas of the aircraft, causing structural fatigue or damage. In severe cases, it can lead to structural failure, endangering flight safety.

[0004] Existing aircraft deformation measurement methods primarily rely on traditional methods, which rely on manual observation and recording at multiple fixed measurement stations. Measurements are then taken from different stations, and offline processing such as coordinate transformation and data fusion is used to obtain measurement results in a unified coordinate system. However, traditional measurement methods often involve frequent human intervention, low measurement efficiency, complex procedures, and poor adaptability. In some special cases, the selection and arrangement of measurement points and measurement stations can be relatively inconvenient, and they cannot meet the requirements for efficient and uniform deformation measurement accuracy, as well as intelligent and automated measurement.

[0005] With the rapid development of science and technology and the aviation manufacturing industry, automated measurement technology has become increasingly mature, especially drone measurement, vehicle-mounted mobile measurement, and backpack mobile measurement. Driven by strong demand for measurement and location services, these technologies and products have continued to develop, solving some of the problems of dynamic measurement technology. However, these technologies have not yet been applied to aircraft deformation measurement.

[0006] In view of the above technical defects of the prior art, there is an urgent need to develop a new aircraft deformation shift measurement method. Summary of the Invention

[0007] The present invention aims to overcome the defects of the prior art and provide an aircraft deformation and station shift measurement method based on a monocular camera and a laser rangefinder. The method has high detection efficiency, small human interference error, low cost, and realizes the automation and intelligent measurement of large-scale objects.

[0008] In order to achieve the above object, the present invention provides the following technical solutions:

[0009] A method for measuring aircraft deformation and station shifting based on a monocular camera and a laser rangefinder, characterized by comprising the following steps:

[0010] 1) Building a mobile robot measurement system, the mobile robot measurement system including a measurement robot and a monocular camera and a laser rangefinder installed on the measurement robot;

[0011] 2) using the monocular camera and the laser rangefinder to obtain distance data between the image of the calibration plate and the feature points to complete the position calibration of the monocular camera and the laser rangefinder;

[0012] 3) Establishing the SLAM measurement map of the measurement robot. The map coordinate system of the SLAM measurement map is fixed to the global coordinate system, and the AMCL positioning module of the measurement robot gives the transformation matrix T between the robot coordinate system and the map coordinate system. R,M ;

[0013] 4) The measurement robot measures the target point on the fuselage of the aircraft placed in the measurement field at a random position in the measurement field to determine the transformation matrix T between the robot coordinate system and the aircraft coordinate system P,R ;

[0014] 5) Based on T R,M and T P,R , the transformation matrix T between the aircraft coordinate system and the map coordinate system is obtained from formula (1) P,M :

[0015] T P,M =T R,M T P,R (1);

[0016] 6) Select multiple optimal measurement points around the aircraft and obtain the coordinates S of the optimal measurement points in the aircraft coordinate system. i , and based on the T P,M , obtain the coordinates M of the optimal measurement point in the map coordinate system through coordinate transformation n ;

[0017] 7) The measuring robot calculates the coordinates M of the optimal measuring point in the map coordinate system. n Perform multi-point path planning and move to the position of the optimal measurement point in sequence to measure the aircraft deformation.

[0018] Preferably, in step 1), the measuring robot is equipped with a pan-tilt platform that can rotate freely horizontally and vertically within a certain range, the monocular camera and the laser rangefinder are fixedly connected to the pan-tilt platform, and the optical axis of the monocular camera and the optical axis of the laser rangefinder are parallel to the ground, and the lens plane of the monocular camera is coplanar with the laser emission plane of the laser rangefinder.

[0019] Compared with the prior art, the aircraft deformation and station shift measurement method based on a monocular camera and a laser rangefinder of the present invention has one or more of the following beneficial technical effects:

[0020] 1. Compared with the existing fixed multi-station aircraft deformation measurement method, the present invention has high detection efficiency, reduces errors caused by human interference, has a simple measurement process, and is highly adaptable. The path planning of the measurement robot ensures that the measurement task is not affected by the size and shape of the site, and can realize the measurement and detection of large equipment as well as automated and intelligent measurement.

[0021] 2. The present invention designs a method for measuring aircraft deformation and station shifting based on a monocular camera and a laser rangefinder. The monocular camera uses vision to guide the rotation of the pan-tilt head to project the laser point of the laser rangefinder onto the target point for distance measurement. The three-dimensional coordinates of the target point are obtained according to the pitch angle and horizontal angle of the pan-tilt head, realizing the measurement and detection of large-scale equipment.

[0022] 3. Compared with other large-scale measurement equipment, the sensors such as monocular camera and laser rangefinder used in the present invention greatly save costs, and are rich in variety and easy to purchase.

[0023] 4. The present invention uses the multilateral intersection method to calculate the coordinates of the light spot of the laser rangefinder, and converts them into the robot coordinate system. The coordinates are adjusted by the indirect adjustment method, thereby achieving accurate positioning of the measuring robot. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 This is a flow chart of the aircraft deformation shift measurement method based on a monocular camera and a laser rangefinder of the present invention.

[0025] Figure 2 This is a schematic diagram of the calibration of a monocular camera and a laser rangefinder.

[0026] Figure 3 This is a schematic diagram of the robot positioning principle based on photogrammetry of the present invention.

[0027] Figure 4 This is a schematic diagram of station relocation measurement. DETAILED DESCRIPTION

[0028] Before describing in detail any embodiment of the present invention, it should be understood that the present invention is not limited in its application to the construction and arrangement details of the components set forth in the following description or illustrated in the following figures. The present invention is capable of other embodiments and can be practiced or carried out in various ways. In addition, it should be understood that the words and terms used herein are for descriptive purposes and should not be considered restrictive. The use of "including" or "having" and their variations herein is intended to cover the items and their equivalents set forth below and additional items. Unless otherwise specified or limited, the terms "mounted", "connected", "supported" and "coupled" and their variations are used broadly and cover direct mounting and indirect mounting, connection, support and coupling. In addition, "connected" and "coupled" are not limited to physical or mechanical connections or couplings.

[0029] Furthermore, on the first hand, in the disclosure of the present invention, the terms "longitudinal", "transverse", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like to indicate orientations or positional relationships are based on the orientations or positional relationships shown in the accompanying drawings, which are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore the above terms cannot be understood as limitations on the present invention; on the second hand, the term "one" should be understood as "at least one" or "one or more", that is, in one embodiment, the number of an element may be one, while in another embodiment, the number of the element may be multiple, and the term "one" cannot be understood as a limitation on the quantity.

[0030] In view of some defects of the existing fixed multi-station measurement method for deformation measurement of large equipment, the present invention designs an aircraft deformation shifting measurement method based on a monocular camera and a laser rangefinder, which can realize the measurement and detection of large equipment as well as the automation and intelligence of measurement.

[0031] Figure 1 The flowchart of the aircraft deformation shift measurement method based on a monocular camera and a laser rangefinder of the present invention is shown. Figure 1 As shown, the aircraft deformation shift measurement method based on a monocular camera and a laser rangefinder of the present invention includes the following steps:

[0032] 1. Build a mobile robot measurement system.

[0033] The mobile robot measurement system includes a measurement robot and a monocular camera and a laser rangefinder installed on the measurement robot.

[0034] In the present invention, the measuring robot serves as a mobile measuring platform, mounted with a pan-tilt platform capable of free horizontal and vertical rotation within a certain range. The monocular camera and laser rangefinder are fixedly connected to the pan-tilt platform, with the optical axes of the monocular camera and the laser rangefinder parallel to the ground, and the lens plane of the monocular camera coplanar with the laser emission plane of the laser rangefinder. To minimize the field of view deviation between the laser rangefinder and the monocular camera, the distance between the monocular camera and the laser rangefinder is as close as possible.

[0035] 2. Use the monocular camera and laser rangefinder to obtain the distance data between the image of the calibration plate and the feature points to complete the position calibration of the monocular camera and the laser rangefinder.

[0036] In the present invention, Figure 2 As shown, the optical center of the monocular camera is taken as the origin, and the direction along the optical axis is Z C Axis and perpendicular to the image plane, X C Axis and Y C The camera coordinate system O is established with the X-axis and Y-axis parallel to the image plane. C X C Y C Z C , take the lower left corner vertex of the calibration plate as the origin, the side closest to the monocular camera as the Z=0 plane, and the positive direction of the Z axis points to the monocular camera to establish the calibration plate coordinate system O W X W Y W Z W , that is, the world coordinate system, and the checkerboard calibration method is used to calibrate the intrinsic parameter K of the monocular camera.

[0037] The calibration plate is used to calibrate the position of the monocular camera and the laser rangefinder. Its essence is to calibrate the coordinates of the laser emission point of the laser rangefinder in the camera coordinate system. The light spot of the laser rangefinder is projected onto the feature point of the calibration plate. The coordinates of the feature point in the world coordinate system are known, and the pixel coordinates of the feature point of the calibration plate can be obtained by an image processing method.

[0038] According to the correspondence between the two sets of coordinates, solve the homography matrix H. The pixel coordinates of the feature points of the calibration plate have the following relationship with the world coordinates:

[0039]

[0040] in, is the pixel coordinate of the feature point, is the coordinate of the feature point in the calibration plate coordinate system (i.e., the world coordinate system), s is the scale factor, and the homography matrix H is a non-singular third-order matrix. After the scale factor s is included in the homography matrix H, the form of H is Then formula (1) can be rewritten as follows:

[0041]

[0042] The homography matrix H is the projection transformation matrix between the points on the calibration plate plane and their corresponding pixels, which can be obtained from the above formula:

[0043]

[0044] Convert equation (3) into matrix form:

[0045] Lh=0 (4)

[0046] in, h=[h 11 h 12 h 13 h 21 h 22 h 23 h 31 h 32 h 33 ] T , the homography matrix H contains h 33 =1 or Constraint, h ij Represents the nine values ​​in the 3rd-order homography matrix H (i=j=1, 2, 3), the unnormalized H matrix, and the nine element values ​​h ij The sum of the squares is equal to 1. When the pixel coordinates and world coordinates of n (n ≥ 4) points on the calibration plane are known, the homography matrix H can be solved.

[0047] Since the pictures taken by the camera contain noise and the position of the points is deviated, the coefficient matrix of formula (5) is solved by the singular value decomposition method to obtain the initial value of H.

[0048] The external parameter matrix M = [r1 r3 r3] between the monocular camera and the calibration plate is calculated through the homography matrix H.

[0049]

[0050] Where λ is the scale factor, K is the camera intrinsic parameter matrix, the camera extrinsic parameter matrix is ​​[r1 r3 r3], and the translation matrix is ​​t. In actual situations, due to the presence of a lot of noise in the image, the solution of Equation (5) will have a large error, and the solution of Equation (5) needs to be optimized.

[0051] Project the light spot of the laser rangefinder onto the characteristic point of the calibration plate, such as Figure 2 As shown. The coordinates of the light spot measured each time in the camera coordinate system The coordinates in the calibration plate coordinate system can be After coordinate transformation, the transformation matrix is ​​[r1r2 r3 t], and the transformation formula is as follows:

[0052] S C =RP W +t (6)

[0053] in, R = [r1r2 r3] and t are obtained from formula (5).

[0054] The distance D measured by the laser rangefinder each time i The coordinates of the laser rangefinder laser emission point in the camera coordinates can be completed by formula (7) with the camera measurement data Calibration,

[0055]

[0056] The data from the laser rangefinder is fused with the data from the monocular camera to perform subsequent measurement tasks.

[0057] 3. Establish the SLAM measurement map of the measurement robot. The map coordinate system of the SLAM measurement map is fixed to the global coordinate system, and the transformation matrix T between the robot coordinate system and the map coordinate system is given by the AMCL positioning module of the measurement robot. R,M .

[0058] In the present invention, the measuring robot first establishes a SLAM measurement map, and the map coordinate system (OX M Y M Z M ) is the global coordinate system, which is fixed and the robot coordinate system is (OX R Y R Z R ), the aircraft coordinate system is (OX P Y P Z P ). Wherein, the transformation relationship between the robot coordinate system and the map coordinate system is T R,M It can be directly given by the AMCL positioning module of the measuring robot.

[0059] 4. The measurement robot measures the target point on the fuselage of the aircraft placed in the measurement field at a random position in the measurement field to determine the transformation matrix T between the robot coordinate system and the aircraft coordinate system P,R .

[0060] A plurality of random positions are selected around the aircraft in the measurement field as measurement points. The field of view of these measurement points must cover the area around the aircraft. At the measurement points, the laser rangefinder is used to measure the target points on the aircraft fuselage one by one.

[0061] Establish the coordinate system O of the laser rangefinder L X L Y L Z L The spherical coordinate expression of the aircraft's fuselage target point in the laser rangefinder coordinate system is P(r, α, β). The data r in the spherical coordinate system is directly measured by the laser rangefinder. The pitch angle α is the angle between the laser emission direction and the Z axis. The horizontal angle β is the angle between the laser beam projected onto the XOY plane and the X axis. The pitch angle α and the horizontal angle β are obtained by the gimbal.

[0062] The distance data r of the laser rangefinder, the pan / tilt angle α, and the horizontal angle β can be used to calculate the coordinates of the aircraft's fuselage target point in the laser rangefinder's coordinate system:

[0063]

[0064] The transformation matrix between the two coordinate systems can be obtained by the coordinates of the aircraft's fuselage target point in the laser rangefinder coordinate system and the corresponding coordinates in the aircraft coordinate system. The transformation relationship between the laser rangefinder coordinate system and the robot coordinate system has been calibrated in advance. Therefore, the transformation matrix T between the robot coordinate system and the aircraft coordinate system can be obtained. P,R .

[0065] 5. Based on T R,M and T P,R , the transformation matrix T between the aircraft coordinate system and the map coordinate system is obtained from formula (9) P,M :

[0066] T P,M =T R,M T P,R (9).

[0067] 6. Select multiple optimal measurement points around the aircraft and obtain the coordinates S of the optimal measurement points in the aircraft coordinate system. i , and based on the T P,M , obtain the coordinates M of the optimal measurement point in the map coordinate system through coordinate transformation n .

[0068] Specifically, the known point coordinates of the target point on the fuselage of the aircraft are used to solve the position coordinates of the measuring robot using the photogrammetry method and perform indirect adjustment to correct the coordinates to complete the positioning of the measuring robot.

[0069] The method of using multilateral measurement to solve the position coordinates of the measuring robot is as follows: the light spot of the laser rangefinder is projected onto the target point on the fuselage of the aircraft to measure the distance. The coordinates of the target point on the fuselage in the aircraft coordinate system are known, and the coordinates of the optical center of the laser rangefinder in the aircraft coordinate system can be solved by the principle of multilateral intersection. Figure 3 shown.

[0070] The coordinates of the fuselage target point n (n = 1, 2, ...) in the aircraft coordinate system are (x n ,y n ,z n ), the coordinates of the optical center point P of the laser rangefinder are (x, y, z), and the distances from point P to the target point n are L n , according to the distance formula, we can get the following equation:

[0071] L n =(x n -x) 2 +(y n -y) 2 +(z n -z) 2 (10)

[0072] Expanding the above formula yields:

[0073]

[0074] Convert equation (11) into the matrix form of AX=B to obtain the approximate coordinates of the optical center of the laser rangefinder.

[0075] Since accidental errors are inevitable in the measurement process, we establish a relationship model between the observed values ​​and the unknown quantities, use the indirect adjustment method to process these observations with accidental errors, and solve the optimal estimate of the coordinates of the optical center point of the laser rangefinder. The specific steps are:

[0076] 1. Use the principle of multilateral intersection to solve the approximate coordinates of the optical center of the laser rangefinder.

[0077] 2. Taylor expand Equation (8) and take the first-order term to obtain:

[0078]

[0079] In the above formula, is the correction factor of the coordinates of the optical center of the laser rangefinder, v n is the residual error of ranging, (x0, y0, z0) are the approximate coordinates of the optical center point P.

[0080] The error equation of formula (12) can be written in matrix form:

[0081] V=AX+L (13)

[0082] Where V=[ν1 ν1 … ν n ],

[0083] 3. Use the least squares principle to find the correction number X and obtain the coordinate adjustment value. The coordinate correction number of point P is:

[0084] X=(A T NA) -1 A T NL (14)

[0085] in,

[0086] The correction number of the coordinates of the optical center point P can be obtained by the above formula. The adjustment coordinates of point P are the approximate coordinates plus the coordinate correction number.

[0087] 4. Use a series of information such as the observation value correction obtained after adjustment to assess accuracy.

[0088] By converting the coordinates of the laser rangefinder after optical center adjustment into the robot coordinate system, the coordinates of the robot in the aircraft coordinate system can be measured, thus achieving precise positioning of the robot in the aircraft coordinate system.

[0089] Compared with the positioning method relying on the sensors carried by the robot itself, the robot positioning method based on photogrammetry of the present invention has higher positioning accuracy.

[0090] 7. The measuring robot determines the coordinates M of the optimal measuring point in the map coordinate system. n Perform multi-point path planning and move to the optimal measurement point in sequence to measure the aircraft deformation. Figure 4 shown.

[0091] Through the above method, no matter how the posture of the indoor aircraft changes, the coordinates of the optimal measurement point can be obtained and moved to the specified position to automatically complete the measurement task.

[0092] Finally, it should be noted that the above embodiments are intended only to illustrate the technical solutions of the present invention and are not intended to limit the scope of protection of the present invention. Those skilled in the art may, based on the principles of the present invention, modify or replace the technical solutions of the present invention with equivalents without departing from the essence and scope of the technical solutions of the present invention.

Claims

1. A method for measuring aircraft deformation and station shifting based on a monocular camera and a laser rangefinder, characterized in that: The following steps are involved: 1) Building a mobile robot measurement system, the mobile robot measurement system including a measurement robot and a monocular camera and a laser rangefinder installed on the measurement robot; 2) using the monocular camera and the laser rangefinder to obtain distance data between the image of the calibration plate and the feature points to complete the position calibration of the monocular camera and the laser rangefinder; 3) Establishing the SLAM measurement map of the measurement robot. The map coordinate system of the SLAM measurement map is fixed to the global coordinate system, and the AMCL positioning module of the measurement robot gives the transformation matrix T between the robot coordinate system and the map coordinate system. R,M ; 4) The measurement robot measures the target point on the fuselage of the aircraft placed in the measurement field at a random position in the measurement field to determine the transformation matrix T between the robot coordinate system and the aircraft coordinate system P,R ; 5) Based on T R,M and T P,R , the transformation matrix T between the aircraft coordinate system and the map coordinate system is obtained from formula (1) P,M : T P,M =T R,M T P,R (1); 6) Select multiple optimal measurement points around the aircraft and obtain the coordinates S of the optimal measurement points in the aircraft coordinate system. i , and based on the T P,M , obtain the coordinates M of the optimal measurement point in the map coordinate system through coordinate transformation n ; 7) The measuring robot calculates the coordinates M of the optimal measuring point in the map coordinate system. n Perform multi-point path planning and move to the position of the optimal measurement point in sequence to measure the aircraft deformation.

2. The method for measuring aircraft deformation and station shifting based on a monocular camera and a laser rangefinder according to claim 1, wherein: In step 1), the measuring robot is equipped with a pan-tilt platform that can rotate freely horizontally and vertically within a certain range, the monocular camera and the laser rangefinder are fixedly connected to the pan-tilt platform, and the optical axis of the monocular camera and the optical axis of the laser rangefinder are parallel to the ground, and the lens plane of the monocular camera is coplanar with the laser emission plane of the laser rangefinder.

Images