A dual-axis inclination sensing device and measurement method based on machine vision

Abstract

The present invention relates to a dual-axis tilt angle sensing device and a measuring method based on machine vision, the device comprising: a body, a camera module, a target module, a gravity ball, and an edge processing module; the body comprises a gravity ball swing bin, a workspace, and a substrate; the camera module comprises a camera module base, a camera module rotation joint, and a camera module; the camera module is connected to the camera module base through the camera module rotation joint, and the camera module is connected to the gravity ball through a connecting rod; the target module comprises a composite graphic luminous target and a target module connecting joint; the composite graphic luminous target is connected to the target base through the target module connecting joint, and the composite graphic luminous target is connected to the gravity ball through a connecting rod; the edge processing module is used to obtain camera images in real time and solve the camera posture, and upload the data to the server in real time. Compared with the prior art, the present invention accurately and stably measures the structural rotation angle, and improves the measurement accuracy and robustness.

Description

Technical Field

[0001] The present invention relates to the technical field of dual-axis inclination angle measurement, and in particular to a dual-axis inclination angle sensing device and a measuring method based on machine vision. Background Art

[0002] There are many methods to monitor the rotation angle and displacement of the structure end, and they can be directly monitored because these parts can easily find relatively fixed points, while the displacement or rotation angle of the measured structure span or cantilever end is difficult to obtain. At present, the displacement of the structure is obtained by non-contact methods, such as long-distance non-contact monitoring methods based on machine vision, but the accuracy and robustness of this technology are greatly affected by the external environment and are not suitable for long-term monitoring. For long-term monitoring, contact monitoring methods can more directly reflect the state of the structure and are more reliable. However, the current contact monitoring methods still have room for improvement in terms of accuracy and long-term monitoring. For example, the measurement methods based on satellite positioning such as GPS or Beidou are not accurate enough to use the monitoring data to evaluate the structural status. Although the connecting pipe method can reflect the overall structural status, its accuracy and dynamic real-time monitoring are difficult to meet the requirements of dynamic evaluation. It is only advantageous in static evaluation. At the same time, for most areas with large annual temperature differences, there is a problem of freezing or evaporation of liquid in the connecting pipe. In addition to the above technologies, many bridges use inclinometers based on MEMS technology to measure the structural inclination to obtain the structural status. Its response speed is very advantageous and the instrument size is very small. However, this type of monitoring method that integrates the monitoring structure acceleration and then solves the structural inclination or displacement has a low accuracy problem in the integration process. Summary of the invention

[0003] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and to provide a dual-axis inclination sensing device and measurement method based on machine vision, which can intuitively monitor the relative displacement between the camera and the target, accurately and stably measure the structural rotation angle, solve the problem of errors caused by acceleration integration, and improve the measurement accuracy and robustness; at the same time, compared with the existing single-axis measurement method, this scheme realizes dual-axis inclination measurement by improving and optimizing the movement mode of the camera module.

[0004] The purpose of the present invention can be achieved by the following technical solutions:

[0005] The present invention provides a dual-axis tilt sensor device based on machine vision, comprising: a body, a camera module, a target module, a gravity ball, and an edge processing module;

[0006] The body includes a gravity ball swinging bin for placing a gravity ball, a working space for placing a camera module and a target module, and a substrate for isolating the camera module, the target module and the gravity ball;

[0007] The camera module includes a camera module base, a camera module rotating joint, and a camera module; the camera module is connected to the camera module base through the camera module rotating joint, and the camera module is connected to the gravity ball through a connecting rod;

[0008] The target module includes a composite graphic luminous target and a target module connecting joint; the composite graphic luminous target is connected to the target base through the target module connecting joint, and the composite graphic luminous target is connected to the gravity ball through a connecting rod;

[0009] The edge processing module is used to acquire camera images and calculate camera posture in real time, and upload the data to the server in real time.

[0010] Furthermore, the composite graphic luminous target includes a composite image luminous icon, which is a circular pattern with a rectangular array, an ARUCO code is set in the circular pattern, and an ARUCO identification code of a different specification from that in the circular pattern is set outside the area of ​​the rectangular array.

[0011] Furthermore, the camera module also includes an installation limit pin, which is used to ensure that no deflection installation error in the horizontal plane will occur when the camera module rotation joint is installed, and no deflection in the horizontal plane during the monitoring process will occur.

[0012] Furthermore, the camera module rotation joint includes a camera module first rotation joint and a camera module second rotation joint which are connected to each other.

[0013] The present invention also provides a dual-axis inclination measurement method, comprising the following steps:

[0014] S1: Connect the camera module to the gravity ball through a connecting rod so that the camera is always kept in a vertical direction. Connect the composite graphic luminous target to the gravity ball through a connecting rod so that the target plane is always perpendicular to the camera line of sight, and neither of them will tilt with the tilt of the object being measured;

[0015] S2: Using the constant distance between the axis of the camera module connecting rod and the axis of the target connecting rod, the displacement of the target in the vertical direction is solved by pnp, and the inverse trigonometric function is used to calculate the inclination angle in the plane formed by the axis of the camera connecting rod, the axis of the target connecting rod and the image center, that is, the inclination angle in the xz plane;

[0016] The camera module always remains vertical in any vertical plane. The target is in the vertical plane formed by the axis of the camera connecting rod, the axis of the target connecting rod and the image center, that is, the yz plane. It will tilt with the tilt of the object being measured. Therefore, the target plane will rotate relative to the camera plane. The relative rotation angle is solved by the pnp (Perspective-n-Point) algorithm to obtain the inclination angle of the structure in the yz plane, thereby realizing the dual-axis inclination measurement of the structure.

[0017] Furthermore, S2 includes: calibrating, detecting and estimating the position and pose of the camera;

[0018] Monitoring includes the following steps:

[0019] A1: Perform image preprocessing on real-time images;

[0020] A2: First, detect the aruco outside the rectangular array area, connect the specific corner points of the aruco to form a detection region of interest ROI;

[0021] A3: In the region of interest obtained in A2, the Aruco code in the circle is identified through the OpenCV library, and the Canny edge detection algorithm is used to highlight the significant features of the image. Then, all the contours in the image are extracted from the edge detected image, and finally the ellipse fitting algorithm is used to identify the ellipse and obtain the parameters of each ellipse.

[0022] Furthermore, the correction method is to calculate the intrinsic parameters and extrinsic parameters of the camera through calibration, and use the obtained correction parameters to calibrate the camera.

[0023] Furthermore, in A3, the contour detection algorithm is the findContours function in OpenCV.

[0024] Furthermore, the pose estimation method is:

[0025] By identifying the circle and the Aruco code inside the circle, since Aruco is uniquely coded, the Euclidean distance between the geometric center of Aruco and the geometric center of the ellipse is used to determine whether the two are in the same position. If they are in the same position, the Aruco ID is assigned to the ellipse, and the 3D coordinates corresponding to the ellipse center are calculated according to the ID number; if the circle at that location is not identified, the Aruco code ID will not be assigned, and the ID will be vacant, that is, the 3D coordinates at that location will not be calculated, thereby achieving a one-to-one correspondence between the 3D coordinates and the image coordinates of the identified ellipse center;

[0026] Using the decoded 3D position corresponding to the tag ID and the detected 2D coordinates, combined with the camera's intrinsic parameters, the PnP (Perspective-n-Point) problem is solved to obtain the camera's position relative to the Aruco code, and then the inclination of the object being measured is obtained;

[0027] Furthermore, the torsional posture of the object under test is determined by solving the rotation angle posture of the camera; the vertical displacement posture of the camera monitored in real time and the distance between the target center and the center of the camera module are taken as constants, and the inverse trigonometric function is used to calculate the tilt posture of the main direction of the object under test, thereby realizing real-time monitoring of the dual-axis tilt posture.

[0028] In fact, by fixing the target module and keeping the camera module as it is, the six degrees of freedom of the camera can be solved, including the pitch angle of the camera (i.e. the tilt angle of the main direction of the object being measured). The present invention designs the target module to be rotatable, and uses inverse trigonometric functions to solve the vertical relative displacement of the two modules. In the case of small deformation, the deflection accuracy outside the plane is low, while the accuracy of the in-plane rotation and displacement of the machine vision is high, so this strategy is used to improve the measurement accuracy. If the measurement accuracy requirement is not high, the solution of fixing the target module can be selected.

[0029] Compared with the prior art, the present invention has the following advantages:

[0030] (1) Realize dual-axis measurement: In this scheme, a device with a multi-directional rotating joint is proposed to realize dynamic monitoring of the rotation angle in multiple directions, overcome the problem that the data of the unidirectional inclination angle is not enough to fully reflect the posture of the measured object, and the dual-axis measurement can solve the problem of on-site installation errors.

[0031] (2) High precision: In this scheme, a composite pattern target is proposed. By identifying the elliptical array and performing ellipse fitting to identify the ellipse center, the accuracy is greatly improved compared with the conventional PNP method that identifies corner points.

[0032] (3) Good robustness: In this scheme, a composite pattern target is proposed to avoid the problem of data drift caused by code loss in the graphic recognition process, greatly improving the long-term stability of data monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 It is a structural schematic diagram of a dual-axis tilt sensor device based on machine vision;

[0034] Figure 2 It is a front view of a dual-axis tilt sensor device based on machine vision;

[0035] Figure 3 A schematic diagram of a composite pattern luminescent target;

[0036] Figure 4 A schematic diagram of a camera module.

[0037] Reference numerals: 1-body; 2-camera module; 3-target module; 4-gravity ball; 5-connecting rod; 6-edge processing module; 7-installation limit pin;

[0038] 101-gravity ball swinging bin; 102-working space; 103-substrate; 201-camera module base; 202-camera module first rotating joint; 203-camera module second rotating joint; 204-camera module; 301-camera module base; 302-composite graphic luminous target; 303-target module connecting joint. DETAILED DESCRIPTION

[0039] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. Component models, material names, connection structures, control methods, algorithms and other features not clearly described in this technical solution are all considered to be common technical features disclosed in the prior art.

[0040] Example 1

[0041] This embodiment provides a dual-axis tilt sensor device based on machine vision, such as Figure 1 , Figure 2 As shown, it includes: a body 1, a camera module 2, a target module 3, a gravity ball 4, and an edge processing module 6;

[0042] The body 1 includes a gravity ball swinging bin 101 for placing a gravity ball 4, a working space 102 for placing a camera module 2 and a target module 3, and a substrate 103 for isolating the camera module 2, the target module 3 and the gravity ball 4;

[0043] The camera module 2 includes a camera module base 201, a camera module rotation joint, and a camera module 204; the camera module 204 is connected to the camera module base 201 through the camera module rotation joint, and the camera module 204 is connected to the gravity ball 4 through a connecting rod 5;

[0044] The target module 3 includes a composite graphic luminous target 302 and a target module connecting joint 303; the composite graphic luminous target 302 is connected to the target base through the target module connecting joint 303, and the composite graphic luminous target 302 is connected to the gravity ball 4 through the connecting rod 5;

[0045] The edge processing module 6 is used to acquire camera images in real time, calculate the camera posture, and upload the data to the server in real time.

[0046] like Figure 3 As shown, in a specific embodiment, the composite graphic luminous target 302 includes a composite image luminous icon, which is a circular pattern with a rectangular array, an ARUCO code is set in the circular pattern, and an ARUCO identification code of a different specification from that in the circular pattern is set outside the area of ​​the rectangular array.

[0047] like Figure 4 As shown, in a specific embodiment, the camera module 2 further includes an installation limit pin 7, which is used to ensure that no deflection installation error in the horizontal plane and no deflection in the horizontal plane during the monitoring process will occur when the camera module rotating joint is installed.

[0048] In a specific implementation, the camera module rotation joint includes a camera module rotation first joint 202 and a camera module rotation second joint 203 that are connected to each other.

[0049] This embodiment also provides a dual-axis inclination measurement method, comprising the following steps:

[0050] S1: Connect the camera module 204 to the gravity ball 4 through the connecting rod 5, so that the camera is always kept in the vertical direction, and connect the composite graphic luminous target 302 to the gravity ball 4 through the connecting rod 5, so that the target plane and the camera line of sight are always perpendicular, and both will not tilt with the tilt of the object being measured;

[0051] S2: Using the constant distance between the axis of the camera module connecting rod and the axis of the target connecting rod, the displacement of the target in the vertical direction is solved by pnp, and the inverse trigonometric function is used to calculate the inclination angle in the plane formed by the axis of the camera connecting rod, the axis of the target connecting rod and the image center, that is, the inclination angle in the xz plane;

[0052] The camera module 204 always maintains a vertical state no matter in which vertical plane, and the target is in a vertical plane formed by the axis of the camera connecting rod, the axis of the target connecting rod and the image center, that is, the yz plane, which will tilt with the tilt of the object being measured. Therefore, the target plane will rotate relative to the camera plane. The relative rotation angle is solved by the pnp (Perspective-n-Point) algorithm to obtain the inclination angle of the structure in the yz plane, thereby realizing the dual-axis inclination measurement of the structure.

[0053] In a specific implementation, S2 includes: calibrating, detecting and estimating the position and posture of the camera;

[0054] Monitoring includes the following steps:

[0055] A1: Perform image preprocessing on real-time images;

[0056] A2: First, detect the aruco outside the rectangular array area, connect the specific corner points of the aruco to form a detection region of interest ROI;

[0057] A3: In the region of interest obtained in A2, the Aruco code in the circle is identified through the OpenCV library, and the Canny edge detection algorithm is used to highlight the significant features of the image. Then, all the contours in the image are extracted from the edge detected image, and finally the ellipse fitting algorithm is used to identify the ellipse and obtain the parameters of each ellipse.

[0058] In a specific implementation, the correction method is to calibrate and calculate the intrinsic and extrinsic parameters of the camera, and use the obtained correction parameters to calibrate the camera.

[0059] In a specific implementation, in A3, the contour detection algorithm is the findContours function in OpenCV.

[0060] In a specific implementation, the pose estimation method is:

[0061] By identifying the circle and the Aruco code inside the circle, since Aruco is uniquely coded, the Euclidean distance between the geometric center of Aruco and the geometric center of the ellipse is used to determine whether the two are in the same position. If they are in the same position, the Aruco ID is assigned to the ellipse, and the 3D coordinates corresponding to the ellipse center are calculated according to the ID number; if the circle at that location is not identified, the Aruco code ID will not be assigned, and the ID will be vacant, that is, the 3D coordinates at that location will not be calculated, thereby achieving a one-to-one correspondence between the 3D coordinates and the image coordinates of the identified ellipse center;

[0062] Using the decoded 3D position corresponding to the tag ID and the detected 2D coordinates, combined with the camera's intrinsic parameters, the PnP (Perspective-n-Point) problem is solved to obtain the camera's position relative to the Aruco code, and then the inclination of the object being measured is obtained;

[0063] In a specific implementation, the torsional posture of the object to be measured is determined by solving the rotation angle posture of the camera; the vertical displacement posture of the camera and the distance between the target center and the camera module center monitored in real time are taken as constants, and the inverse trigonometric function is used to calculate the tilt posture of the main direction of the object to be measured, thereby realizing real-time monitoring of the dual-axis tilt posture.

[0064] In fact, the target module is fixed and the camera module 204 is kept as it is, so that the six degrees of freedom of the camera can be solved, including the pitch angle of the camera (i.e., the tilt attitude of the main direction of the measured object). The present invention designs the target module 3 to be rotatable, and uses the inverse trigonometric function to solve the vertical relative displacement of the two modules. In the case of small deformation, the deflection accuracy outside the plane is low, while the accuracy of the in-plane rotation and displacement of the machine vision is high, so this strategy is adopted to improve the measurement accuracy. If the measurement accuracy requirement is not high, the solution of fixing the target module can be selected.

[0065] Components not described in detail in this embodiment are all existing components that can be purchased through public channels.

[0066] The above description of the embodiments is to facilitate the understanding and use of the invention by those skilled in the art. It is obvious that those skilled in the art can easily make various modifications to these embodiments and apply the general principles described herein to other embodiments without creative work. Therefore, the present invention is not limited to the above embodiments, and improvements and modifications made by those skilled in the art based on the disclosure of the present invention without departing from the scope of the present invention should be within the scope of protection of the present invention.

Claims

1. A dual-axis tilt sensor device based on machine vision, characterized in that: include: A body (1), a camera module (2), a target module (3), a gravity ball (4), and an edge processing module (6); The body (1) comprises a gravity ball swinging bin (101) for placing a gravity ball (4), a working space (102) for placing a camera module (2) and a target module (3), and a base plate (103) for isolating the camera module (2), the target module (3) and the gravity ball (4); The camera module (2) comprises a camera module base (201), a camera module rotating joint, and a camera module (204); the camera module (204) is connected to the camera module base (201) via the camera module rotating joint, and the camera module (204) is connected to the gravity ball (4) via a connecting rod (5); The target module (3) comprises a composite graphic luminous target (302) and a target module connecting joint (303); the composite graphic luminous target (302) is connected to a target base via the target module connecting joint (303), and the composite graphic luminous target (302) is connected to a gravity ball (4) via a connecting rod (5); The edge processing module (6) is used to acquire camera images in real time, calculate camera postures, and upload the data to the server in real time; The composite graphic luminous target (302) comprises a composite image luminous icon, which is a circular pattern with a rectangular array, an Aruco code is arranged in the circular pattern, and an Aruco identification code of a different specification from that in the circular pattern is arranged outside the area of ​​the rectangular array; The camera module (2) further comprises an installation limit pin (7), wherein the installation limit pin (7) is used to ensure that a deflection installation error in a horizontal plane and a deflection in a horizontal plane during monitoring will not occur when the camera module rotation joint is installed.

2. A dual-axis tilt sensor device based on machine vision according to claim 1, characterized in that: The camera module rotation joint comprises a camera module rotation first joint (202) and a camera module rotation second joint (203) which are connected to each other.

3. A dual-axis inclination angle measurement method using the sensing device according to any one of claims 1 to 2, characterized in that: The following steps are involved: S1: Connecting the camera module (204) to the gravity ball (4) via a connecting rod (5) so that the camera is always kept in a vertical direction, and connecting the composite graphic luminous target (302) to the gravity ball (4) via a connecting rod (5) so that the target plane is always perpendicular to the camera line of sight, and neither of them will tilt with the tilt of the object being measured; S2: Using the constant distance between the axis of the camera module connecting rod and the axis of the target connecting rod, the displacement of the target in the vertical direction is solved by pnp, and the inverse trigonometric function is used to calculate the inclination angle in the plane formed by the axis of the camera connecting rod, the axis of the target connecting rod and the image center, that is, the inclination angle in the xz plane; The target is in the vertical plane formed by the axis of the camera connecting rod, the axis of the target connecting rod and the image center, that is, the yz plane. It will tilt with the tilt of the object being measured. Therefore, the target plane will rotate relative to the camera plane. The relative rotation angle is solved by the pnp algorithm to obtain the inclination angle of the structure in the yz plane, thereby realizing the dual-axis inclination measurement of the structure.

4. A dual-axis inclination angle measurement method according to claim 3, characterized in that: S2 includes: calibration, detection and pose estimation of the camera; The monitoring comprises the following steps: A1: Perform image preprocessing on real-time images; A2: First, detect the aruco outside the rectangular array area, connect the specific corner points of the aruco to form a detection region of interest ROI; A3: In the region of interest obtained in A2, the Aruco code in the circle is identified through the OpenCV library, and the Canny edge detection algorithm is used to highlight the significant features of the image. Then, all the contours in the image are extracted from the edge detected image, and finally the ellipse fitting algorithm is used to identify the ellipse and obtain the parameters of each ellipse.

5. The measuring method of a dual-axis tilt sensor device according to claim 4, characterized in that: The correction method is to calibrate and calculate the intrinsic parameters and extrinsic parameters of the camera, and use the obtained correction parameters to calibrate the camera.

6. A dual-axis inclination angle measurement method according to claim 4, characterized in that: The pose estimation method is: By identifying the circle and the Aruco code inside the circle, the Euclidean distance between the geometric center of Aruco and the geometric center of the ellipse is used to determine whether the two are in the same position. If they are in the same position, the Aruco ID is assigned to the ellipse, and the 3D coordinates corresponding to the ellipse center are calculated according to the ID number; if the circle at that location is not identified, the Aruco code ID will not be assigned, and the ID will be vacant, that is, the 3D coordinates at that location will not be calculated, thereby achieving a one-to-one correspondence between the 3D coordinates and the image coordinates of the identified ellipse center; Using the decoded 3D position corresponding to the marker ID and the detected 2D coordinates, combined with the intrinsic parameters of the camera, the PnP problem is solved to obtain the position and posture of the camera relative to the Aruco code, and then the inclination angle of the object being measured is obtained.

7. A dual-axis inclination angle measurement method according to claim 6, characterized in that: The torsion posture of the object under test is determined by the calculated rotation angle posture of the camera; the vertical displacement posture of the camera monitored in real time and the distance between the target center and the camera module center are taken as constants, and the inverse trigonometric function is used to calculate the tilt posture of the main direction of the object under test, thereby realizing real-time monitoring of the dual-axis tilt posture.

Images