Marine dynamic calibration method and system of multi-degree-of-freedom binocular vision system

Abstract

The invention discloses an offshore dynamic calibration method and system of a multi-degree-of-freedom binocular vision system. The calibration method comprises the following steps: S1, establishing a ship body coordinate system on a ship; s2, internal reference calibration of the binocular camera; s3, calibrating a first camera coordinate system and a double-axis tilt angle sensor coordinate system; s4, performing static calibration on the first camera coordinate system and the world coordinate system; s5, calibrating external parameters of the binocular camera; and S6, dynamically calibrating the coordinate systems of the first camera and the second camera and the world coordinate system. According to the invention, special requirements of the marine dynamic environment can be met through innovative design, accurate calibration of the binocular camera in the marine dynamic environment can be realized, and high precision and high reliability of the binocular stereoscopic vision system in the marine environment can be ensured, so that special challenges in the marine environment can be adapted, and the calibration accuracy of the binocular camera is improved. According to the invention, high-precision measurement of a target object in a marine dynamic environment can be realized, so that the precision and safety of marine operation are improved, and important technical support is provided for ocean engineering.

Description

Technical Field

[0001] The present invention relates to the field of machine vision and measurement technology, and in particular to a marine dynamic calibration method and system for a multi-degree-of-freedom binocular vision system. Background Art

[0002] With the rapid development of science and technology, the application of camera technology is becoming more and more extensive in various fields, especially in the fields of marine engineering and computer vision. In these fields, binocular stereo vision cameras have attracted much attention because they can provide rich three-dimensional spatial information. It acquires images through two cameras shooting from different angles and reconstructs the depth information of the photographed target for detecting and tracking moving targets. The calibration accuracy of binocular stereo vision cameras greatly affects the accuracy of binocular measurement, so it has become the focus of research and development.

[0003] At present, the commonly used calibration method in binocular ranging is the Zhang Zhengyou calibration method. This method calibrates the camera based on a 2D plane target. By capturing multiple pictures of the calibration plate and matching the actual points (world coordinates) with the points on the picture (pixel coordinates) one by one, the relationship between the world coordinates and the pixel coordinates is determined. However, the Zhang Zhengyou calibration method requires photographing the calibration plate in the use scene, which is difficult to achieve in some environments; especially in the marine environment, due to the dynamic nature of the ocean and the lack of stable reference points, traditional calibration methods face huge challenges.

[0004] In addition, the complexity of the marine environment, such as the movement of ships and the fluctuation of the sea surface, places higher requirements on the calibration of the vision system. In this environment, the calibration process needs to consider not only the parameters of the camera itself, but also the position and attitude of the camera relative to the mobile platform (such as a ship). Therefore, traditional calibration methods are often difficult to adapt to the special requirements of the marine environment, resulting in limited accuracy and reliability in actual marine applications.

[0005] In view of this, the development of a binocular stereo vision system that can adapt to the dynamic marine environment for high-precision calibration and measurement has become an urgent need in the fields of marine engineering and computer vision. Summary of the invention

[0006] The technical problem to be solved by the present invention is to provide a method and system for dynamic calibration of a multi-degree-of-freedom binocular vision system at sea in view of the deficiencies in the above-mentioned prior art. The core of the present invention is to solve the challenges unique to the marine environment, such as the movement of ships and the lack of stable reference points, and to achieve high-precision calibration of the binocular camera system under such dynamic conditions. This technology is of great significance for improving the accuracy and reliability of binocular stereo vision systems in applications such as marine construction, marine exploration, and environmental monitoring. The system can significantly improve the efficiency and safety of marine operations. Through precise measurement and data analysis, the system can provide important visual assistance for offshore construction, optimize the construction process, and reduce the impact of the marine environment on construction operations.

[0007] To achieve the above-mentioned object, the technical solution adopted by the present invention is as follows: In a first aspect of the present invention, a method for dynamic calibration of a multi-degree-of-freedom binocular vision system at sea is provided, the method being used to calibrate a binocular camera working on a ship at sea, the binocular camera comprising a first camera and a second camera, the first camera being provided with a dual-axis tilt sensor; the targets used for calibration in the method comprising a static target and a dynamic target, the static target being fixedly arranged in a land calibration field, the dynamic target being floatingly arranged at sea and spaced apart from the ship;

[0008] The method comprises the following steps:

[0009] S1. Establishing a hull coordinate system on the ship: using the satellite station difference method to establish a series of control points on the hull of the ship, obtaining the geodetic coordinates of these control points, and then defining the hull coordinate system based on these control points;

[0010] S2. Internal calibration of the binocular camera: In a land calibration field, the internal calibration of the binocular camera is performed using Zhang Zhengyou's chessboard calibration method to obtain the internal parameters and distortion coefficients of the first camera and the second camera;

[0011] S3. Calibration of the first camera coordinate system and the dual-axis tilt sensor coordinate system: In the land calibration field, the positions of the dual-axis tilt sensor and the first camera are fixed, and the three-dimensional information and pixel information of the static target in the public field of view are respectively extracted using the total station and the first camera. Multiple PnP calibrations are performed, and the angle output of the dual-axis tilt sensor during each calibration process is recorded, and then the rotation matrix from the dual-axis tilt sensor coordinate system to the first camera coordinate system is obtained. ;

[0012] S4. Static calibration of the first camera coordinate system and the world coordinate system:

[0013] S4-1, keep the ship relatively still with respect to the standard ground, install the first camera and the dual-axis tilt sensor whose position relationship has been calibrated on the hull, and install the second camera at other positions;

[0014] S4-2. Establishing the world coordinate system ;

[0015] S4-3, set the total station on the hull, and then use the total station and the first camera to respectively extract the three-dimensional points and pixel points of the dynamic target in the public field of view, and calculate the initial rotation matrix between the world coordinate system and the first camera coordinate system in a PnP calibration manner , translation matrix , and record the dual-axis tilt sensor at this time Axis and Axis angle output , ;

[0016] S5. External parameter calibration of binocular camera: Keep the ship still, take multiple images of dynamic targets in the public field of view of the binocular camera, and solve the rotation matrix R and translation matrix from the first camera coordinate system to the second camera coordinate system. ;

[0017] S6, dynamic calibration of the first camera and the second camera coordinate system and the world coordinate system: release the stillness of the ship, take multiple images of dynamic targets in the public field of view of the binocular camera, and for each shot of the binocular camera, record the angle output of the dual-axis tilt sensor, set The first The dual-axis tilt sensor Axis and The angular outputs of the axes are , , ; Then get the rotation matrix from the world coordinate system to the first camera coordinate system , translation matrix , and then the rotation matrix obtained in step S5 , translation matrix Calculate the rotation matrix from the world coordinate system to the second camera coordinate system , translation matrix , complete the calibration of the binocular camera on the hull. After the calibration is completed, the binocular camera can accurately measure the coordinates, inclination (including inclination azimuth and angle), elevation and plane position of marine structures from the ship, and the measurement solution can adapt to the dynamic changes in the marine environment.

[0018] Preferably, step S1 specifically includes:

[0019] A differential global positioning system is used to establish a series of control points on the hull of the ship, and the geodetic coordinates of these control points are obtained, and then a hull coordinate system is defined based on these control points, and the hull coordinate system is aligned with the geodetic coordinate system;

[0020] The origin of the hull coordinate system is set as the geometric center of the hull, and the three axes are set as follows: the X-axis is set to the direction from the bow to the stern, the Y-axis is set to the direction across the width of the hull and perpendicular to the X-axis, and the Y-axis is set to the direction perpendicular to the hull surface.

[0021] Preferably, step S3 specifically includes:

[0022] S3-1. In a land calibration field, the positions of the dual-axis tilt sensor and the first camera are fixed, a total station is set in the land calibration field, and then the total station and the first camera are used to respectively extract the three-dimensional information and pixel information of the static target in the public field of view, and multiple PnP calibrations are performed, and the angle output of the dual-axis tilt sensor during each calibration process is recorded;

[0023] Dual-axis inclination sensor Axis and The shaft output angle is , , there is a standard horizontal plane ;

[0024] Set standard level around Axis rotation Angle, then around Axis rotation Angle to get the plane parallel to the current dual-axis tilt sensor coordinate system, let Axis rotation Angle and dual-axis tilt sensor Shaft output angle Equal, we get the following angle conversion relationship:

[0025] ;

[0026] S3-2, let the rotation matrix from the total station coordinate system to the first camera coordinate system be , the rotation matrix from the dual-axis tilt sensor coordinate system to the first camera coordinate system is ;

[0027] set up The inverse matrix , ; then we get The formula of the element:

[0028] ;

[0029] S3-3. Use a static target to perform PnP calibration on the total station and the first camera to obtain the rotation matrix from the total station coordinate system to the first camera coordinate system. , and record the angle output of the dual-axis tilt sensor each time , , and then solve the rotation matrix from the dual-axis inclination coordinate system to the first camera coordinate system .

[0030] Preferably, step S4 specifically includes:

[0031] S4-1, keep the ship relatively still with respect to the standard ground, install the first camera and the dual-axis tilt sensor whose position relationship has been calibrated on the hull, and install the second camera at other positions;

[0032] S4-2. Establishing the world coordinate system , the world coordinate system The axis is always perpendicular to the standard horizontal plane and vertically upward. The axis is parallel to the projection of the line connecting the centers of the first camera and the second camera on the standard horizontal plane. Axis vector and The direction of the cross product of the axis vector is Axis direction;

[0033] S4-3, set the total station on the hull, and then use the total station and the first camera to respectively extract the three-dimensional points and pixel points of the dynamic target in the public field of view, and calculate the initial rotation matrix between the world coordinate system and the first camera coordinate system in the static calibration in a PnP calibration manner , translation matrix , and record the dual-axis tilt sensor at this time Axis and Axis angle output , , as the initial angle output of the dual-axis inclination sensor, and then according to the angle conversion relationship in step S3, we get , ;

[0034] .

[0035] Preferably, step S6 specifically includes:

[0036] S6-1. Release the stillness of the ship, take multiple images of dynamic targets in the public field of view of the binocular camera, and for each shot of the binocular camera, record the angle output of the dual-axis tilt sensor. The first The dual-axis tilt sensor Axis and The angular outputs of the axes are , , ;

[0037] S6-2, combined with the initial angle output obtained in step S4 to obtain , , get the rotation matrix from the world coordinate system to the first camera coordinate system in dynamic calibration , translation matrix , and then the rotation matrix obtained in step S5 , translation matrix Calculate the rotation matrix from the world coordinate system to the second camera coordinate system , translation matrix , complete the calibration of the binocular camera on the hull.

[0038] Preferably, in step S6, the rotation matrix from the world coordinate system to the first camera coordinate system is The specific calculation method is:

[0039] S6-2-1, let the three-dimensional point in the dual-axis tilt sensor coordinate system be , ,in, , represents the three-dimensional point coordinates in the dual-axis tilt sensor coordinate system under the static calibration condition of step S4, recorded as the static three-dimensional point ;

[0040] When , it represents the dynamic calibration state of step S6, the binocular camera The coordinates of the three-dimensional point in the dual-axis tilt sensor coordinate system during the first shooting are recorded as dynamic three-dimensional points ;

[0041] S6-2-2, Static 3D Points To dynamic 3D point The conversion process is as follows;

[0042] ;

[0043] By the following rotation matrix relationship:

[0044]

[0045] Among them, Rsw represents the rotation matrix between the world coordinate system and the dual-axis tilt sensor coordinate system during the dynamic calibration process;

[0046] The rotation matrix from the world coordinate system to the first camera coordinate system is:

[0047] .

[0048] According to a second aspect of the present invention, a marine dynamic calibration system for a multi-degree-of-freedom binocular vision system is provided. The system uses the method described above to calibrate a binocular camera working on a marine vessel. The binocular camera includes a first camera and a second camera. The first camera is provided with a dual-axis tilt sensor.

[0049] Preferably, the marine dynamic calibration system of the multi-degree-of-freedom binocular vision system comprises:

[0050] Total station, used for distance and angle measurements during calibration;

[0051] Targets include static targets and dynamic targets, wherein the static targets are fixedly arranged in a land calibration field, and the dynamic targets are floatingly arranged at sea and spaced apart from the ship;

[0052] The checkerboard calibration plate is used to perform internal calibration of the binocular camera using Zhang Zhengyou's checkerboard calibration method;

[0053] And a processing module is used to analyze and process the data received from the binocular camera and the total station, and display the analysis and processing results.

[0054] Preferably, the processing module is a computer, which includes a computer host for performing data analysis and processing and a display screen for displaying the analysis and processing results.

[0055] A third aspect of the present invention provides a binocular stereo vision system that can adapt to the dynamic marine environment, which includes the marine dynamic calibration system and a binocular camera as described above.

[0056] The beneficial effects of the present invention are:

[0057] The present invention provides a marine dynamic calibration method and a calibration system for a multi-degree-of-freedom binocular vision system, and further provides a binocular stereo vision system equipped with the calibration system and capable of adapting to a marine dynamic environment; the binocular camera calibration method and system provided by the present invention can adapt to the special requirements of a marine dynamic environment through innovative design, can realize accurate calibration of binocular cameras in a marine dynamic environment, can ensure high precision and high reliability of the binocular stereo vision system in a marine environment, thereby being able to adapt to special challenges in a marine environment, and ultimately enabling the binocular stereo vision system using the calibration scheme to achieve high-precision measurement of targets in a marine dynamic environment, thereby improving the precision and safety of marine operations, and providing important technical support for marine engineering. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 It is a schematic diagram of a marine dynamic calibration system of a multi-degree-of-freedom binocular vision system in Example 1 of the present invention;

[0059] Figure 2 is a schematic structural diagram of a first camera in Embodiment 1 of the present invention;

[0060] Figure 3 This is a flow chart of the method for dynamic calibration of a binocular vision system at sea in Embodiment 1 of the present invention;

[0061] Figure 4 Schematic diagram of camera calibration principle in Embodiment 1 of the present invention;

[0062] Figure 5 Schematic diagram of the geometric relationship between coordinate systems in Example 1 of the present invention.

[0063] Description of reference numerals:

[0064] 1-first camera; 2-camera lens; 3-two-dimensional high-precision turntable; 4-synchronous controller; 5-dual-axis tilt sensor and camera fixing platform; 6-first camera data transmission line; 7-first camera power line; 8-total station; 9-total station 8 bracket; 10-total station data transmission line; 11-computer host; 12-display screen; 13-second camera; 14-second camera data transmission line; 15-second camera power line; 16-target; 17-small boat; 18-offshore pile foundation platform; 19-offshore pile foundation; 2 0-camera lens; 21-camera fixing device; 22-camera control module; 23-camera data interface; 24-camera power interface; 25-dual-axis tilt sensor; 26-dual-axis tilt sensor power interface; 27-dual-axis tilt sensor data interface; 28-turntable; 29-camera synchronization control device; 30-data interface; 31-power interface; 32-chessboard calibration plate; 33-imaging position; 34-imaging plane; 35-camera optical center; 36-camera main optical axis; 37-image plane principal point; DETAILED DESCRIPTION

[0065] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0066] Example 1

[0067] A marine dynamic calibration system for a multi-DOF binocular vision system is disclosed. The system is used to calibrate a binocular camera working on a marine ship. The binocular camera comprises a first camera 1 and a second camera 13. The first camera 1 is provided with a dual-axis tilt sensor 25.

[0068] Reference Figure 1 The system comprises:

[0069] Total station 8, used for accurate distance and angle measurement during calibration;

[0070] The target 16 includes a static target and a dynamic target. The static target is fixedly set in the land calibration field and is used for the intrinsic calibration of the camera and determining the attitude relationship between the dual-axis tilt sensor 25 and the camera; the dynamic target is floated on the sea and spaced apart from the ship and is used for the static calibration of the first camera coordinate system and the world coordinate system and the extrinsic calibration of the binocular camera;

[0071] A checkerboard calibration plate 31, used to perform internal reference calibration of the binocular camera using Zhang Zhengyou's checkerboard calibration method;

[0072] And a processing module, which is used to analyze and process the data received from the binocular camera and the total station 8, and display the analysis and processing results; in this embodiment, the processing module is a computer, which includes a computer host 11 for performing data analysis and processing and a display screen 12 for displaying the analysis and processing results.

[0073] Refer to Figure 2 , wherein the first camera 1 includes a camera lens 20, a two-dimensional high-precision turntable 3, a synchronous controller 4, a two-axis tilt sensor 25, a camera control module 22, a camera fixing device 21, etc. The camera lens 20 is the imaging part of the first camera 1 and is responsible for capturing images; the two-dimensional high-precision turntable 3 is used to control the rotation angle of the first camera 1; the synchronous controller 4 is used to control the rotation of the two-dimensional high-precision turntable 3 and record the posture data of the camera and the two-axis tilt sensor 25 at the same time; the camera control module 22 is used to control the first camera 1; the camera fixing device 21 is used to keep the camera lens 20 fixed. The bottom of the first camera 1 is provided with a two-axis tilt sensor 25 and a camera fixing platform 5, which are used to firmly attach the two-axis tilt sensor 25 and the camera to the corresponding position of the hull to ensure shooting stability. The first camera 1 is provided with a camera data interface 23 and a camera power interface 24. The first camera 1 is connected to the processing module through the first camera data transmission line 6 for transmitting the captured image data and posture data; and is connected to the power supply through the first camera power line 7 for continuous power supply of the device. The dual-axis inclination sensor 25 is provided with a dual-axis inclination sensor power interface 26 and a dual-axis inclination sensor data interface 27 .

[0074] A camera synchronization control device 29 is also provided at the bottom of the first camera 1. In this embodiment, the camera synchronization control device 29 adopts a 32-bit microprocessor and controls the camera to take pictures by using a differential level trigger method; a data interface 30 and a power interface 31 are provided thereon.

[0075] The second camera 13 may adopt the same structure as the first camera 1, or a different structure, which at least includes a camera lens responsible for capturing images, a second camera data transmission line 14 for connecting the second camera 13 and a processing module, and a second camera power line 15 for connecting the second camera 13 and a power source.

[0076] The total station 8 is supported by a total station bracket 9 to ensure its stability; the total station 8 and the processing module are connected via a total station data transmission line 10 for synchronous measurement and transmission of measurement data.

[0077] The dynamic target is supported by a small boat 17, which is floated on the sea surface and spaced apart from the ship, so as to move the target 16 on the water and provide support for dynamic calibration.

[0078] Figure 1 Also illustrated are an offshore pile foundation platform 18 and an offshore column pile 19. The offshore pile foundation platform 18 is a structure in the marine environment and is used for calibration and testing of the present invention; the offshore column pile 19 is a target for calibration and measurement, represents a typical element in marine construction, and is installed to support the offshore pile foundation platform 18.

[0079] Reference Figure 3 In this embodiment, the calibration method of the marine dynamic calibration system of the multi-degree-of-freedom binocular vision system includes the following steps:

[0080] S1. Establish control points on the ship for control measurement and define the ship coordinate system:

[0081] A differential global positioning system is used to establish a series of control points on the hull of the ship, and the geodetic coordinates of these control points are obtained, and then a hull coordinate system is defined based on these control points, and the hull coordinate system is aligned with the geodetic coordinate system;

[0082] The origin of the hull coordinate system is set as the geometric center of the hull, and the three axes are set as follows: the X-axis is set to the direction from the bow to the stern, the Y-axis is set to the direction across the width of the hull and perpendicular to the X-axis, and the Y-axis is set to the direction perpendicular to the hull surface.

[0083] In the dynamic calibration of the binocular vision system at sea, the first step is to establish control points on the construction survey vessel and perform control measurements to define the hull coordinate system. The core of this step is to use the differential global positioning system (DGPS) to accurately establish control points on the ship and define the hull coordinate system based on these control points. The use of DGPS is to overcome the errors existing in the traditional GPS system and improve the positioning accuracy to the centimeter level, which is crucial for accurate engineering measurements in dynamic environments at sea.

[0084] Specifically, in this embodiment, first, a series of control points are established on the ship through DGPS. These control points are calibrated relative to known geographic coordinates to ensure that they have accurate references in the global coordinate system. The working principle of DGPS includes receiving GPS signals at a reference station at a known position, calculating its error with the theoretical position, and sending this error information to a nearby DGPS receiver, which uses this information to correct its own GPS data, thereby improving positioning accuracy.

[0085] Next, based on these control points, define a hull coordinate system. This coordinate system uses a three-dimensional Cartesian coordinate system, with its origin set at a fixed reference point on the hull, such as the geometric center of the hull. The three axes of the coordinate system are: X-axis (longitudinal axis, along the bow to stern direction), Y-axis (transverse axis, across the width of the hull), and Z-axis (vertical axis, perpendicular to the hull plane). The setting of this coordinate system needs to take into account the actual structure and operational requirements of the hull, and align with the geodetic coordinate system (such as WGS-84) to ensure the consistency and accuracy of the coordinate system.

[0086] Given that the hull may move or rotate during offshore operations, dynamic adjustment of the ship's coordinate system becomes necessary. This is achieved by real-time monitoring of the hull's position and attitude, usually using high-precision measurement technologies such as inertial navigation systems. Inertial navigation systems determine the position, velocity, and attitude of an object by tracking acceleration and angular velocity. When used in conjunction with the DGPS system, the hull's coordinate system can be dynamically tracked and adjusted in real time to ensure that it reflects the hull's real-time position and attitude at all times. This combination not only improves measurement accuracy, but also enhances the system's adaptability to dynamic offshore environments.

[0087] Through the above steps, a coordinate system is established that is aligned with the global coordinate system and can dynamically reflect the real-time status of the hull, laying the foundation for the accurate calibration of the multi-degree-of-freedom binocular vision system in the dynamic environment at sea. This step is the key to achieving high-precision and high-reliability dynamic visual measurement at sea, and provides the necessary prerequisite for the implementation of this patent.

[0088] S2. Perform internal calibration of the binocular camera in the land calibration field based on Zhang Zhengyou's calibration method:

[0089] In the land calibration field, the Zhang Zhengyou chessboard calibration method is used to perform internal parameter calibration on the binocular camera to obtain the internal parameters and distortion coefficients of the first camera 1 and the second camera 13.

[0090] Zhang Zhengyou calibration method is used as follows Figure 4 For the checkerboard calibration plate 31 shown, after obtaining an image of the calibration plate, the pixel coordinates (u, v) of each corner point can be obtained using a corresponding image detection algorithm.

[0091] like Figure 4 , is a schematic diagram of the camera calibration principle, wherein the checkerboard calibration plate 31 is a standardized tool for camera calibration, usually having alternating black and white squares, which is used to help determine the internal parameters of the camera, such as focal length and lens distortion.

[0092] Reference Figure 5 , which is a schematic diagram of the geometric relationship between coordinate systems, where:

[0093] Imaging plane 34: The plane on which the camera's internal optical system projects the external scene. It is the area where the image is formed and is usually located at the camera sensor. The imaging position 33 is also shown in the figure.

[0094] Camera optical center 35: The center point of the camera lens 20 system, which is the point where all light entering the lens converges and is also the reference point of the imaging system.

[0095] Camera principal optical axis 36: A straight line from the camera optical center 35 perpendicular to the imaging plane 34, representing the camera's line of sight and an important reference for the camera's field of view and imaging process.

[0096] Image plane principal point 37: a point on the imaging plane 34 where the principal optical axis intersects the imaging plane 34. In camera calibration, this point is a key reference for locating the origin of the image coordinate system.

[0097] The world coordinate system is recorded as , the camera coordinate system is recorded as , the image pixel coordinate system is recorded as , the image physical coordinate system is recorded as .

[0098] Zhang Zhengyou's calibration method fixes the world coordinate system on the chessboard, so the physical coordinates of any point on the chessboard are W=0. Since the world coordinate system of the calibration board is pre-defined by humans, the size of each grid on the calibration board is known, and the physical coordinates of each corner point in the world coordinate system can be calculated (U, V, W=0).

[0099] During the calibration process, this information will be used: the pixel coordinates of each corner point (u, v) and the physical coordinates of each corner point in the world coordinate system (U, V, W=0) to calibrate the camera and obtain the camera's intrinsic parameter matrix and distortion parameters.

[0100] In dynamic scenes at sea, during the measurement process of the binocular vision system, the observation direction of the binocular camera is toward the outside of the hull. It is difficult to arrange control points on the outside of the hull and because the targets outside the hull will move relative to the hull under marine conditions, it is impossible to arrange enough control points to perform external calibration of the binocular system. To solve this problem, the present invention adopts an innovative method to perform external calibration on the binocular camera; first, the binocular vision camera is arranged according to the construction situation, and the camera direction and baseline distance are adjusted; secondly, the binocular camera is installed on the fixed observation pier of the hull, and it is noted that the observation conditions of the observation pier need to meet the distance of engineering measurement monitoring and the imaging range requirements of the target on the boat 17. After installation, the observation pier, camera and hull are assumed to be in a relatively rigid relationship; thirdly, by setting up a total station 8 on a known control point of the ship, and connecting it to the binocular vision system through a data link to a computer, a target is placed on the boat 17 arranged in front of the construction ship, and the total station 8 accurately measures the coordinates of the target feature points. At the same time, the binocular lens captures the image of the target, and the coordinates of the target feature points are measured by the total station 8. The corresponding feature point coordinates in the image are used for external parameter calibration to obtain the precise external orientation elements of the camera in the hull coordinate system.

[0101] S3. Calibration of the first camera coordinate system and the dual-axis tilt sensor coordinate system:

[0102] S3-1, in the land calibration field, fix the positions of the dual-axis tilt sensor 25 and the first camera 1, set the total station 8 in the land calibration field, and then use the total station 8 and the first camera 1 to respectively extract the three-dimensional information and pixel information of the static target in the public field of view, perform multiple PnP calibrations, and record the angle output of the dual-axis tilt sensor 25 during each calibration process;

[0103] A dual-axis tilt sensor 25 is provided Axis and The shaft output angle is , , there is a standard horizontal plane ;

[0104] Set standard level around Axis rotation Angle, then around Axis rotation Angle to get the plane parallel to the current dual-axis tilt sensor coordinate system, let Axis rotation Angle and dual axis tilt sensor 25 Shaft output angle Equal, we get the following angle conversion relationship:

[0105] ;

[0106] S3-2, let the rotation matrix from the total station 8 coordinate system to the first camera coordinate system be , the rotation matrix from the dual-axis tilt sensor coordinate system to the first camera coordinate system is ;

[0107] set up The inverse matrix , ; then we get The formula of the element:

[0108] ;

[0109] S3-3, use the static target to perform PnP calibration on the total station 8 and the first camera 1, and obtain the rotation matrix from the coordinate system of the total station 8 to the coordinate system of the first camera , and record the angle output of the dual-axis tilt sensor 25 each time , , and then solve the rotation matrix from the dual-axis inclination coordinate system to the first camera coordinate system .

[0110] S4. Static calibration of the first camera coordinate system and the world coordinate system:

[0111] S4-1, keep the ship relatively still with respect to the standard ground, stably install the first camera 1 and the dual-axis tilt sensor 25 whose position relationship has been calibrated on the ship, and install the second camera 13 at other appropriate positions;

[0112] S4-2. Establishing the world coordinate system , the world coordinate system The axis is always perpendicular to the standard horizontal plane and vertically upward. The axis is parallel to the projection of the center line of the first camera 1 and the second camera 13 on the standard horizontal plane, Axis vector and The direction of the cross product of the axis vector is Axis direction;

[0113] S4-3, set the total station 8 on the hull, and then use the total station 8 and the first camera 1 to respectively extract the three-dimensional points and pixel points of the dynamic target in the public field of view, and calculate the initial rotation matrix between the world coordinate system and the first camera coordinate system in the static calibration in the PnP calibration method. , translation matrix , and record the dual-axis tilt sensor 25 at this time Axis and Axis angle output , , as the initial angle output of the dual-axis tilt sensor 25, and then according to the angle conversion relationship in step S3, obtain , ;

[0114] .

[0115] The PnP (Perspective-n-Point) calibration method used is a well-known and commonly used method in the field of computer vision and photogrammetry. It is mainly used to calculate the camera's posture (rotation and translation) from several known world coordinate points and their corresponding image coordinate points. This method uses at least three three-dimensional points and their two-dimensional projections in the image to determine the spatial position and orientation of the camera, and obtains the initial rotation matrix by solving the correspondence between the camera coordinate system and the world coordinate system. , translation matrix .

[0116] S5. External parameter calibration of binocular camera:

[0117] Keep the ship still, take multiple images of dynamic targets in the common field of view of the binocular camera, and solve the rotation matrix R and translation matrix from the first camera coordinate system to the second camera coordinate system. .

[0118] S6, dynamic calibration of the first camera and second camera coordinate systems and the world coordinate system:

[0119] S6-1. Release the ship from its stationary state, and take multiple images of dynamic targets in the public field of view of the binocular camera. For each shot of the binocular camera, record the angle output of the dual-axis tilt sensor 25. Set The first The dual-axis tilt sensor 25 is Axis and The angular outputs of the axes are , , ;

[0120] S6-2, combined with the initial angle output obtained in step S4 to obtain , , get the rotation matrix from the world coordinate system to the first camera coordinate system in dynamic calibration , translation matrix , and then the rotation matrix obtained in step S5 , translation matrix Calculate the rotation matrix from the world coordinate system to the second camera coordinate system , translation matrix , complete the calibration of the binocular camera on the hull. This provides the necessary camera pose information support for further accurate measurement of parameters such as the inclination, elevation and plane position of the target object (such as marine piles).

[0121] In step S6, the rotation matrix from the world coordinate system to the first camera coordinate system is The specific calculation method is:

[0122] S6-2-1, let the three-dimensional point in the dual-axis tilt sensor coordinate system be , ,in, , represents the three-dimensional point coordinates in the dual-axis tilt sensor coordinate system under the static calibration condition of step S4, recorded as the static three-dimensional point ;

[0123] When , it represents the dynamic calibration state of step S6, the binocular camera The coordinates of the three-dimensional point in the dual-axis tilt sensor coordinate system during the first shooting are recorded as dynamic three-dimensional points ;

[0124] S6-2-2, Static 3D Points To dynamic 3D point The conversion process is as follows;

[0125] ;

[0126] By the following rotation matrix relationship:

[0127]

[0128] Among them, Rsw represents the rotation matrix between the world coordinate system and the dual-axis tilt sensor coordinate system during the dynamic calibration process;

[0129] The rotation matrix from the world coordinate system to the first camera coordinate system is:

[0130] ;

[0131] In step S6, the translation matrix from the world coordinate system to the first camera coordinate system is The calculation method is:

[0132] According to the translation matrix between the world coordinate system and the first camera coordinate system obtained by the PnP calibration method in step S4 In this step of dynamic calibration, the output of the dual-axis inclination sensor 25 is used to , right Update to get the translation matrix .

[0133] Example 2

[0134] A binocular stereo vision system capable of adapting to a dynamic marine environment includes a marine dynamic calibration system and a binocular camera as in Example 1. The marine dynamic calibration system and method in Example 1 are used to calibrate the binocular camera operating in a dynamic marine environment, thereby ensuring high precision and high reliability of the binocular stereo vision system in a marine environment, thereby adapting to special challenges in a marine environment, and achieving high-precision measurement of targets in a dynamic marine environment, thereby improving the precision and safety of marine operations.

[0135] The above description is only a specific implementation of the present application, so that those skilled in the art can understand or implement the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest range consistent with the principles and novel features applied for herein.

Claims

1. A method for dynamic calibration of a multi-degree-of-freedom binocular vision system at sea, characterized in that: The method is used to calibrate a binocular camera working on a ship at sea, the binocular camera includes a first camera and a second camera, the first camera is provided with a dual-axis tilt sensor; the targets used for calibration in the method include a static target and a dynamic target, the static target is fixedly set in a land calibration field, and the dynamic target is floated on the sea and spaced apart from the ship; The method comprises the following steps: S1. Establishing a hull coordinate system on the ship: using the satellite station difference method to establish a series of control points on the hull of the ship, obtaining the geodetic coordinates of these control points, and then defining the hull coordinate system based on these control points; S2. Internal calibration of the binocular camera: In a land calibration field, the internal calibration of the binocular camera is performed using Zhang Zhengyou's chessboard calibration method to obtain the internal parameters and distortion coefficients of the first camera and the second camera; S3. Calibration of the first camera coordinate system and the dual-axis tilt sensor coordinate system: In the land calibration field, the positions of the dual-axis tilt sensor and the first camera are fixed, and the three-dimensional information and pixel information of the static target in the public field of view are respectively extracted using the total station and the first camera. Multiple PnP calibrations are performed, and the angle output of the dual-axis tilt sensor during each calibration process is recorded, and then the rotation matrix from the dual-axis tilt sensor coordinate system to the first camera coordinate system is obtained. ; S4. Static calibration of the first camera coordinate system and the world coordinate system: S4-1, keep the ship relatively still with respect to the standard ground, install the first camera and the dual-axis tilt sensor whose position relationship has been calibrated on the hull, and install the second camera at other positions; S4-2. Establishing the world coordinate system ; S4-3, set the total station on the hull, and then use the total station and the first camera to respectively extract the three-dimensional points and pixel points of the dynamic target in the public field of view, and calculate the initial rotation matrix between the world coordinate system and the first camera coordinate system in a PnP calibration manner , translation matrix , and record the dual-axis tilt sensor at this time Axis and Axis angle output , ; S5. External parameter calibration of binocular camera: Keep the ship still, take multiple images of dynamic targets in the public field of view of the binocular camera, and solve the rotation matrix R and translation matrix from the first camera coordinate system to the second camera coordinate system. ; S6, dynamic calibration of the first camera and the second camera coordinate system and the world coordinate system: release the stillness of the ship, take multiple images of dynamic targets in the public field of view of the binocular camera, and for each shot of the binocular camera, record the angle output of the dual-axis tilt sensor, set The first The dual-axis tilt sensor Axis and The angular outputs of the axes are , , ; Then get the rotation matrix from the world coordinate system to the first camera coordinate system , translation matrix , and then the rotation matrix obtained in step S5 , translation matrix Calculate the rotation matrix from the world coordinate system to the second camera coordinate system , translation matrix , complete the calibration of the binocular camera on the hull; Step S1 specifically includes: A differential global positioning system is used to establish a series of control points on the hull of the ship, and the geodetic coordinates of these control points are obtained, and then a hull coordinate system is defined based on these control points, and the hull coordinate system is aligned with the geodetic coordinate system; The origin of the hull coordinate system is set as the geometric center of the hull, and the three axes are set as follows: the X-axis is set to the direction from the bow to the stern, the Y-axis is set to the direction across the width of the hull and perpendicular to the X-axis, and the Y-axis is set to the direction perpendicular to the hull surface.

2. The marine dynamic calibration method of a multi-degree-of-freedom binocular vision system according to claim 1, characterized in that: Step S3 specifically includes: S3-1. In a land calibration field, the positions of the dual-axis tilt sensor and the first camera are fixed, a total station is set in the land calibration field, and then the total station and the first camera are used to respectively extract the three-dimensional information and pixel information of the static target in the public field of view, and multiple PnP calibrations are performed, and the angle output of the dual-axis tilt sensor during each calibration process is recorded; Dual-axis inclination sensor Axis and The shaft output angle is , , there is a standard horizontal plane ; Set standard level around Axis rotation Angle, then around Axis rotation Angle to get the plane parallel to the current dual-axis tilt sensor coordinate system, let Axis rotation Angle and dual-axis tilt sensor Shaft output angle Equal, we get the following angle conversion relationship: ； S3-2, let the rotation matrix from the total station coordinate system to the first camera coordinate system be , the rotation matrix from the dual-axis tilt sensor coordinate system to the first camera coordinate system is ; set up The inverse matrix , ; then we get The formula of the element: ； S3-3. Use a static target to perform PnP calibration on the total station and the first camera to obtain the rotation matrix from the total station coordinate system to the first camera coordinate system. , and record the angle output of the dual-axis tilt sensor each time , , and then solve the rotation matrix from the dual-axis inclination coordinate system to the first camera coordinate system .

3. The marine dynamic calibration method of the multi-degree-of-freedom binocular vision system according to claim 2 is characterized in that: Step S4 specifically includes: S4-1, keep the ship relatively still with respect to the standard ground, install the first camera and the dual-axis tilt sensor whose position relationship has been calibrated on the hull, and install the second camera at other positions; S4-2. Establishing the world coordinate system , the world coordinate system The axis is always perpendicular to the standard horizontal plane and vertically upward. The axis is parallel to the projection of the line connecting the centers of the first camera and the second camera on the standard horizontal plane. Axis vector and The direction of the cross product of the axis vector is Axis direction; S4-3, set the total station on the hull, and then use the total station and the first camera to respectively extract the three-dimensional points and pixel points of the dynamic target in the public field of view, and calculate the initial rotation matrix between the world coordinate system and the first camera coordinate system in the static calibration in the PnP calibration method , translation matrix , and record the dual-axis tilt sensor at this time Axis and Axis angle output , , as the initial angle output of the dual-axis inclination sensor, and then according to the angle conversion relationship in step S3, we get , ; 。 4. The marine dynamic calibration method of a multi-degree-of-freedom binocular vision system according to claim 3 is characterized in that: Step S6 specifically includes: S6-1. Release the stillness of the ship, take multiple images of dynamic targets in the public field of view of the binocular camera, and for each shot of the binocular camera, record the angle output of the dual-axis tilt sensor. The first The dual-axis tilt sensor Axis and The angular outputs of the axes are , , ; S6-2, combined with the initial angle output obtained in step S4 to obtain , , get the rotation matrix from the world coordinate system to the first camera coordinate system in dynamic calibration , translation matrix , and then the rotation matrix obtained in step S5 , translation matrix Calculate the rotation matrix from the world coordinate system to the second camera coordinate system , translation matrix , complete the calibration of the binocular camera on the hull.

5. The marine dynamic calibration method of a multi-degree-of-freedom binocular vision system according to claim 4, characterized in that: In step S6, the rotation matrix from the world coordinate system to the first camera coordinate system is The specific calculation method is: S6-2-1, let the three-dimensional point in the dual-axis tilt sensor coordinate system be , ,in, , represents the three-dimensional point coordinates in the dual-axis tilt sensor coordinate system under the static calibration condition of step S4, recorded as the static three-dimensional point ; When , it represents the dynamic calibration state of step S6, the binocular camera The coordinates of the three-dimensional point in the dual-axis tilt sensor coordinate system during the first shooting are recorded as dynamic three-dimensional points ; S6-2-2. Static 3D points To dynamic 3D point The conversion process is as follows; ； By the following rotation matrix relationship: ； Among them, Rsw represents the rotation matrix between the world coordinate system and the dual-axis tilt sensor coordinate system during the dynamic calibration process; The rotation matrix from the world coordinate system to the first camera coordinate system is: 。 6. A marine dynamic calibration system for a multi-degree-of-freedom binocular vision system, characterized in that: The system calibrates a binocular camera operating on a marine vessel using the method described in any one of claims 1 to 5, wherein the binocular camera comprises a first camera and a second camera, and the first camera is provided with a dual-axis tilt sensor.

7. The marine dynamic calibration system of the multi-DOF binocular vision system according to claim 6, characterized in that: The system includes: Total station, used for distance and angle measurements during calibration; Targets include static targets and dynamic targets, wherein the static targets are fixedly arranged in a land calibration field, and the dynamic targets are floatingly arranged at sea and spaced apart from the ship; The checkerboard calibration plate is used to perform internal calibration of the binocular camera using Zhang Zhengyou's checkerboard calibration method; And a processing module is used to analyze and process the data received from the binocular camera and the total station, and display the analysis and processing results.

8. The marine dynamic calibration system of the multi-DOF binocular vision system according to claim 7, characterized in that: The processing module is a computer, which includes a computer host for performing data analysis and processing and a display screen for displaying the analysis and processing results.

9. A binocular stereo vision system capable of adapting to the dynamic environment at sea, characterized in that: It comprises a marine dynamic calibration system and a binocular camera as described in any one of claims 6 to 8.

Images