System and method for measuring the height of a ship based on a gimbal and a binocular camera all-around extended view

Abstract

The application discloses a system and method for measuring the height of a ship based on a holder and a binocular camera, a camera calibration module calibrates the internal and external parameters of the binocular camera; the holder is controlled to rotate left and right, so that the ship appears in the field of view of the binocular camera; an image processing operation is performed on the image of the left camera after internal parameter correction by a highest point detection module, and the holder is controlled by a holder control module to adjust the pitch and left and right directions, so that the highest point of the ship appears in the field of view; a binocular stereo matching module performs binocular stereo matching on the corrected images of the left and right cameras, a depth map and a three-dimensional point cloud are generated based on the coordinate system of the left camera, a data correction module corrects the generated three-dimensional point cloud according to the pitch angle of the binocular camera at the moment, and the real height of the ship is obtained according to the three-dimensional point cloud of the corrected highest point of the ship. The holder is used to control the binocular camera to measure, and the height of the ship at different positions with different heights can be quickly and accurately measured.

Description

Technical Field

[0001] This invention relates to the field of ship height measurement technology, specifically a system and method for omnidirectional panoramic measurement of ship height based on a gimbal and binocular camera. Background Technology

[0002] With the continuous increase in inland waterway shipping volume, the phenomenon of inland waterway vessels exceeding height limits and violating regulations is also increasing, seriously affecting the navigation safety, efficiency, and economic benefits of inland waterways, and hindering the healthy development of the shipping economy.

[0003] Currently, the main technology for measuring the height of vessels in inland waterways involves setting up multiple sensors at different heights or using a single sensor to measure at different heights. However, this method has certain limitations and inconveniences.

[0004] Traditional binocular vision measurement methods have a limited field of view, making them unsuitable for measuring ships at different heights. Furthermore, measurements cannot be taken when the ship leaves the binocular camera's field of view. In practice, multiple binocular cameras are often installed at different heights and locations to measure ships at varying altitudes and positions, increasing equipment costs and inconvenience.

[0005] Therefore, there is an urgent need for a new method for measuring ship height to enable convenient measurement of the height of ships at different heights and locations. Summary of the Invention

[0006] To address the aforementioned technical problems, this invention provides a system and method for omnidirectional panoramic measurement of ship height based on a gimbal and binocular camera. By controlling the binocular camera with a gimbal, the height of ships at different heights and locations can be measured quickly and accurately. Compared with traditional ship height measurement methods, this method reduces the use of sensors and greatly improves the convenience of measurement, thus having broad application prospects.

[0007] Therefore, the present invention adopts the following technical solution:

[0008] This invention provides a system for omnidirectional panoramic measurement of ship altitude based on a gimbal and binocular camera, the system comprising:

[0009] A binocular camera used for acquiring images of ships;

[0010] Connected to a stereo camera, this module is used to calibrate the stereo camera, including a camera calibration module that solves for the intrinsic parameter matrix, distortion coefficient, and extrinsic parameter matrix of the left and right cameras.

[0011] A highest point detection module that connects to the camera calibration module to detect the highest point;

[0012] A gimbal control module that connects to the highest point detection module to control the pitch, rotation, and left-right adjustment of the gimbal, ensuring that the highest point of the ship appears within the field of view of the binocular camera.

[0013] Connected to the highest point detection module, the binocular stereo matching module performs binocular stereo matching on the images acquired by the left and right cameras based on the intrinsic and extrinsic parameters of the binocular camera, and generates a disparity map and corresponding 3D point cloud with the left camera as the reference.

[0014] Tilt sensor used to detect the tilt angle of gimbal;

[0015] A data correction module that connects to the binocular stereo matching module and the tilt sensor to perform pitch and translation corrections on the 3D point cloud based on the camera coordinate system of the left camera.

[0016] Connected to the data correction module, this is a height calculation module used to calculate the ship's altitude.

[0017] Furthermore, on a gimbal with two binocular cameras placed horizontally, the tilt sensor is positioned near the left camera.

[0018] Furthermore, the camera calibration module adopts the Zhang Zhengyou calibration method, which involves taking pictures of the checkerboard pattern at different positions and using a calibration algorithm to solve for the camera parameters, thereby obtaining the camera's intrinsic parameter matrix and distortion coefficients, as well as the extrinsic parameter matrix between the left and right cameras.

[0019] Furthermore, the highest point detection module performs intrinsic parameter calibration on the camera, takes the image with intrinsic parameter correction acquired by the left camera, performs background modeling on it, extracts the moving ship target, and performs binarization processing on the image, setting the pixel where the ship target is located to 1 and the rest to 0.

[0020] Then, an opening operation is performed on the image to remove isolated noise points;

[0021] Traverse the pixels from bottom to top. If a row of pixels is all 0, then the row below that row contains the highest point, and retrieve the coordinates of the highest point. If no row of pixels is all 0, control the gimbal to rotate upwards until a row of pixels is detected to be all 0.

[0022] Furthermore, the data correction module performs pitch correction, including:

[0023] The camera coordinate system rotates counterclockwise around the x-axis. If it rotates by α degrees around the x-axis at this time, let the coordinates of the highest point in the 3D point cloud be P1(X′). L ,Y′ L Z′ L The pitch-corrected coordinates are P0(X). L Y LZ L If the pitch correction value is 0, then the highest point coordinate Y is 0. L For: Y L =Y′ L cosα+Z′ L sinα;

[0024] Simultaneously, as the gimbal rotates, it will tilt slightly around the z-axis of the coordinate system. If it tilts counterclockwise by θ degrees around the z-axis, then the corrected coordinates of the highest point Y will be... L For: Y L =-X′ L cosαsinθ+Y′ L cosαcosθ+Z′ L sinα.

[0025] Furthermore, the data correction module performs translation correction, including:

[0026] Translation describes a linear shift in the coordinate system containing point P, moving by t along the negative coordinate axes. x , t y , t z The distance is such that the coordinates of P′ in the new coordinate system are achieved by adding a translation vector t to P:

[0027]

[0028] When the gimbal rotates upwards by α degrees, similar triangles can be used to determine that the gimbal's rotation axis corresponds to a rotation of α degrees. Let the distance from the camera's optical center to the gimbal's rotation axis be Δh, and the resulting translation t in the Y-axis direction at this time... y for:

[0029] t y =Δh*(1-cosα);

[0030] The translation t generated in the Z-axis direction at this time z for:

[0031] t z =Δh*sinα;

[0032] Let the coordinates of the highest point in the 3D point cloud at this moment be P1(X′). L ,Y′ L Z′ L The coordinates after translation correction are P0(X). L Y L Z L If the coordinates are ), then the coordinates after translation correction are:

[0033]

[0034] Furthermore, the data correction module comprehensively considers pitch correction and translation correction, assuming the coordinates of the highest point in the 3D point cloud before correction are P1(X′). L ,Y′ L Z′ L The corrected coordinates are P0(X). L Y L Z L If the Y-coordinate of the corrected highest point is then... L For: Y L =-X′ L cosαsinθ+Y′ L cosαcosθ+Z′ L sinα-Δh*(1-cosα).

[0035] This invention also provides a method for omnidirectional panoramic measurement of ship height based on a gimbal and binocular camera, the method comprising:

[0036] S1. Perform internal and external parameter calibration on the stereo camera, and calculate the intrinsic parameter matrix, distortion coefficient, and extrinsic parameter matrix of the left and right cameras respectively.

[0037] S2. Control the gimbal to rotate left and right so that the ship appears in the field of view of the binocular camera;

[0038] S3. Correct the image acquired by the camera at this time according to the camera's intrinsic parameters;

[0039] S4. Detect the highest point through the highest point detection module, and adjust the pitch and lateral of the control gimbal based on the highest point to make the highest point of the ship appear in the field of view and obtain the coordinates of the highest point, while tracking the highest point of the ship.

[0040] S5. Based on the intrinsic and extrinsic parameters of the binocular camera, perform binocular stereo matching on the images acquired by the left and right cameras, and generate a disparity map and corresponding 3D point cloud with the left camera as the reference.

[0041] S6. Perform pitch correction on the generated 3D point cloud based on the current pitch angle of the binocular camera;

[0042] S7. Perform translation correction on the generated 3D point cloud based on the current pitch angle of the binocular camera and the radius of the gimbal rotation axis.

[0043] S8. Obtain the true height of the ship based on the corrected 3D point cloud of the ship's highest point.

[0044] Compared with the prior art, the present invention has the following beneficial effects:

[0045] This invention uses a gimbal to control the left, right, and pitch rotation of a binocular camera, thereby obtaining a wider field of view to meet the needs of measuring ships at different heights and positions. Simultaneously, it corrects for the rotation of the gimbal and the resulting translational displacement, resulting in more accurate and precise corrections. Image processing algorithms control the gimbal's pitch rotation, automatically finding the ship's highest point and ensuring it falls within the binocular camera's field of view. This invention offers advantages such as real-time performance, high precision, and ease of operation, and can be effectively applied in the field of ship altitude measurement. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 This is a system block diagram of an embodiment of the present invention for omnidirectional panoramic measurement of ship height based on a gimbal and binocular camera;

[0048] Figure 2 This is an example diagram of sensor installation in an embodiment of the present invention;

[0049] Figure 3 This is a flowchart illustrating a method for omnidirectional panoramic measurement of ship height based on a gimbal and binocular camera, as described in an embodiment of the present invention.

[0050] Figure 4 This is a flowchart of the image processing algorithm in an embodiment of the present invention;

[0051] Figure 5 This is a schematic diagram of pitch and rotation in an embodiment of the present invention;

[0052] Figure 6 This is a schematic diagram of the camera coordinate system before and after pitch correction in an embodiment of the present invention;

[0053] Figure 7 This is a schematic diagram of translation correction in an embodiment of the present invention;

[0054] In the diagram, 1. Binocular camera; 2. Tilt sensor; 3. Camera calibration module; 4. Gimbal control module; 5. Highest point detection module; 6. Binocular stereo matching module; 7. Data correction module. Detailed Implementation

[0055] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0056] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0057] like Figure 1 As shown, this embodiment of the invention provides a system for omnidirectional extended-view measurement of ship altitude based on a gimbal and a binocular camera 1. The system includes: a binocular camera 1, a tilt sensor 2, a camera calibration module 3, a gimbal control module 4, a highest point detection module 5, a binocular stereo matching module 6, a data correction module 7, and an altitude calculation module. During system installation, as... Figure 2 As shown, two binocular cameras 1 are placed horizontally on a gimbal, with the tilt sensor 2 positioned near the left camera. Wherein:

[0058] Binocular camera 1, used to acquire images of the ship;

[0059] Camera calibration module 3 is connected to stereo camera 1 and is used to calibrate stereo camera 1, including solving for the intrinsic parameter matrix, distortion coefficient, and extrinsic parameter matrix of the left and right cameras. Zhang Zhengyou's calibration method is adopted. By photographing a checkerboard pattern at different positions and using the calibration algorithm to solve for the camera parameters, the intrinsic parameter matrix and distortion coefficient of the camera, as well as the extrinsic parameter matrix (rotation matrix and translation matrix) between the left and right cameras, are obtained.

[0060] The highest point detection module 5 is connected to the camera calibration module 3 and is used to detect the highest point. Specifically, after the camera's intrinsic parameters are calibrated, the image acquired by the left camera after intrinsic parameter correction is taken, and background modeling is performed on it to extract the moving ship target. The image is then binarized, setting the pixels containing the ship target to 1 and the rest to 0. Then, an opening operation is performed on the image to remove isolated noise points. The pixels are traversed from bottom to top. If a row of pixels is all 0, the row below it is the row containing the highest point, and the coordinates of the highest point are extracted. If no row of pixels is all 0, the gimbal is controlled to rotate upwards until a row of pixels all 0 is detected. After obtaining the coordinates of the highest point in the image, the gimbal is rotated left and right to position the highest point close to the center of the image. At the same time, the gimbal is rotated left and right to track the highest point of the ship as it moves, keeping it close to the center of the image until a new ship arrives or the gimbal rotates to its limit position.

[0061] The gimbal control module 4 is connected to the highest point detection module 5, which controls the gimbal to rotate left and right so that the ship appears in the field of view of the binocular camera 1. The highest point detection module 5 also controls the gimbal to tilt and rotate left and right so that the highest point of the ship appears in the field of view of the binocular camera 1.

[0062] The binocular stereo matching module 6 is connected to the highest point detection module 5. Based on the intrinsic and extrinsic parameters of the binocular camera 1, the images acquired by the left and right cameras are matched in a binocular stereo manner, and a disparity map and the corresponding three-dimensional point cloud are generated with the left camera as the reference.

[0063] Tilt sensor 2 is used to detect the tilt angle of the gimbal;

[0064] The data correction module 7 is connected to the binocular stereo matching module 6 and the tilt sensor 2, and performs pitch correction and translation correction on the three-dimensional point cloud with the camera coordinate system of the left camera as the reference.

[0065] The altitude calculation module is connected to the data correction module 7 and is used to calculate the ship's altitude.

[0066] The working process of the above system is as follows:

[0067] First, the camera calibration module 3 is used to calibrate the intrinsic and extrinsic parameters of the binocular camera 1, obtaining information such as the intrinsic and extrinsic parameters of the binocular camera 1. Then, the gimbal is controlled to rotate left and right so that the ship appears within the field of view of the binocular camera 1. The highest point detection module 5 processes the image of the left camera after intrinsic parameter correction, and the gimbal control module 4 controls the gimbal to adjust its pitch and left and right to make the highest point of the ship appear within the field of view, obtaining the two-dimensional coordinates of the highest point at this time, and simultaneously controlling the gimbal to track the highest point of the ship. The image acquired by the camera at this time is corrected according to the camera's intrinsic parameters, and then the binocular stereo matching module 6 performs binocular stereo matching on the corrected images of the left and right cameras, generating a depth map and a 3D point cloud based on the coordinate system of the left camera. The data correction module 7 performs pitch correction on the generated 3D point cloud according to the pitch angle of the binocular camera 1 at this time, and then performs translation correction on the generated 3D point cloud according to the pitch angle obtained by the tilt sensor 2 of the binocular camera 1 at this time and the radius of the gimbal rotation axis. The true height of the ship is obtained from the corrected 3D point cloud of the ship's highest point.

[0068] Based on the above system, such as Figure 3 As shown in the figure, an embodiment of the present invention provides a method for omnidirectional panoramic measurement of ship height based on a gimbal and binocular camera. First, the binocular camera is calibrated. Then, the highest point of the ship is found through the gimbal control module and the highest point detection module. Next, a 3D point cloud of the highest point is generated through binocular stereo matching. The 3D point cloud is then corrected based on data transmitted from the tilt sensor, and finally, the ship's height is calculated. Specifically, the method includes the following steps:

[0069] S1. Perform internal and external parameter calibration on the stereo camera, and calculate the intrinsic parameter matrix, distortion coefficient, and extrinsic parameter matrix of the left and right cameras respectively.

[0070] S2. Control the gimbal to rotate left and right so that the ship appears in the field of view of the binocular camera.

[0071] S3. Correct the image acquired by the camera at this time according to the camera's intrinsic parameters.

[0072] S4. The highest point is detected by the highest point detection module. Based on the highest point, the gimbal is controlled to adjust the pitch and lateral angles so that the highest point of the ship appears in the field of view and the coordinates of the highest point are obtained. At the same time, the highest point of the ship is tracked.

[0073] Specifically, after calibrating the camera's intrinsic parameters, the image acquired by the left camera after intrinsic parameter correction is used for background modeling to extract the moving ship target. The image is then binarized, setting the pixels containing the ship target to 1 and the rest to 0. Next, an opening operation is performed to remove isolated noise points. The pixels are traversed from bottom to top. If a row of pixels is all 0, the row below it contains the highest point, and its coordinates are extracted. If no row of pixels is all 0, the gimbal is rotated upwards until a row of all pixels is detected. After obtaining the coordinates of the highest point in the image, the gimbal is rotated left and right to position the highest point near the center of the image. Simultaneously, the gimbal is rotated left and right to track the ship's highest point as it moves, keeping it close to the center of the image until a new ship arrives or the gimbal reaches its limit. The algorithm flow is as follows: Figure 4 As shown.

[0074] S5. Based on the intrinsic and extrinsic parameters of the binocular camera, perform binocular stereo matching on the images acquired by the left and right cameras, and generate a disparity map and the corresponding 3D point cloud with the left camera as the reference.

[0075] S6. Perform pitch correction on the generated 3D point cloud based on the current pitch angle of the binocular camera.

[0076] After rotating the three-dimensional coordinate axes, the relationship after rotation around the x-axis is:

[0077] Y′ r =Y r cosα-Z r sinα (1)

[0078] Z′ r =Y r cosα+Z r sinα (2)

[0079]

[0080] Similarly, the relationship around the y-axis and around the z-axis is as follows:

[0081]

[0082]

[0083] Multiplying the rotation components in the three directions yields the rotation matrix between the two three-dimensional coordinates:

[0084]

[0085] Pitch correction: When the gimbal rotates upwards, as... Figure 5As shown, the camera coordinate system rotates counterclockwise around the x-axis. If it rotates by α degrees around the x-axis at this time, let the coordinates of the highest point in the 3D point cloud be P1(X′). L ,Y′ L Z′ L The pitch-corrected coordinates are P0(X). L Y L Z L If the pitch correction value is 0, then the highest point coordinate Y is 0. L for:

[0086] Y L =Y′ L cosα+Z′ L sinα (7)

[0087] Simultaneously, as the gimbal rotates, it will tilt slightly around the z-axis of the coordinate system. If it tilts counterclockwise by θ degrees around the z-axis, then the corrected coordinates of the highest point Y will be... L for:

[0088] Y L =-X′ L cosαsinθ+Y′ L cosαcosθ+Z′ L sinα (8)

[0089] Schematic diagram of coordinate system before and after pitch correction as follows Figure 6 As shown.

[0090] S7. Perform translation correction on the generated 3D point cloud based on the current pitch angle of the binocular camera and the radius of the gimbal rotation axis.

[0091] Translation correction: Translation describes a linear shift in the coordinate system containing point P, moving by t along the negative coordinate axes. x , t y , t z If the distance is such that the coordinates of P′ in the new coordinate system are such that the distance is such that the coordinates of P′ can be obtained by adding a translation vector t to P:

[0092]

[0093] like Figure 7 As shown, when the gimbal rotates upwards by α degrees, similar triangles can be obtained, indicating that the gimbal's rotation axis rotates by α degrees accordingly. Let the distance from the camera's optical center to the gimbal's rotation axis be Δh, and the resulting translation t in the Y-axis direction at this time... y for:

[0094] t y =Δh*(1-cosα) (10)

[0095] The translation t generated in the Z-axis direction at this timez for:

[0096] t z =Δh*sinα (11)

[0097] Let the coordinates of the highest point in the 3D point cloud at this moment be P1(X′). L ,Y′ L Z′ L The coordinates after translation correction are P0(X). L Y L Z L If the coordinates are ), then the coordinates after translation correction are:

[0098]

[0099] S8. Obtain the true height of the ship based on the corrected 3D point cloud of the ship's highest point.

[0100] Taking into account both pitch and translation corrections, let the coordinates of the highest point in the 3D point cloud before correction be P1(X′). L ,Y′ L Z′ L The corrected coordinates are P0(X). L Y L Z L If the Y-coordinate of the corrected highest point is then... L for:

[0101] Y L =-X′ L cosαsinθ+Y′ L cosαcosθ+Z′ L sinα-Δh*(1-cosα) (13)

[0102] The ship's true height can be calculated based on the corrected coordinates of the highest point.

[0103] Let S be the height of the camera above the water surface at this moment, then the ship's clearance height H at this moment is:

[0104] H = Y L +S (14)

[0105] In the above embodiments, the measurement is performed by controlling a binocular camera with a gimbal, which can quickly and accurately measure the height of ships at different heights and positions. Compared with traditional ship height measurement methods, it can reduce the use of sensors and greatly improve the convenience of measurement, and has broad application prospects.

[0106] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A system for omnidirectional panoramic measurement of ship height based on a gimbal and binocular camera (1), characterized in that, The system includes: A binocular camera used to acquire images of ships (1); Connected to the binocular camera (1), it is used to calibrate the binocular camera (1), including a camera calibration module (3) for solving the intrinsic parameter matrix, distortion coefficient and extrinsic parameter matrix of the left and right cameras. Connected to the camera calibration module (3), the highest point detection module (5) is used to detect the highest point. Connected to the highest point detection module (5), the gimbal control module (4) controls the pitch and rotation of the gimbal and adjusts it left and right so that the highest point of the ship appears in the field of view of the binocular camera (1). Connected to the highest point detection module (5), the binocular stereo matching module (6) performs binocular stereo matching on the images acquired by the left and right cameras according to the intrinsic and extrinsic parameters of the binocular camera (1) and generates a disparity map and corresponding three-dimensional point cloud based on the left camera. Tilt sensor (2) used to detect the tilt angle of gimbal; A data correction module (7) is connected to the binocular stereo matching module (6) and the tilt sensor (2) to perform pitch correction and translation correction on the three-dimensional point cloud based on the camera coordinate system of the left camera. Connected to the data correction module (7), it is a height calculation module used to calculate the ship's height; The highest point detection module (5) calibrates the camera's intrinsic parameters, takes the image of the left camera after intrinsic parameter correction, models the background of the image, extracts the moving ship target, and performs binarization on the image, setting the pixel where the ship target is located to 1 and the rest to 0. Then, an opening operation is performed on the image to remove isolated noise points; Traverse the pixels from bottom to top. If a row of pixels is all 0, then the row below that row contains the highest point, and retrieve the coordinates of the highest point. If no row of pixels is all 0, control the gimbal to rotate upwards until a row of pixels is detected to be all 0. The data correction module (7) performs pitch correction, including: Camera coordinate system rotation x If the axis rotates counterclockwise, then at this time... x Axis rotation Let the coordinates of the highest point in the 3D point cloud be (degrees). The coordinates after pitch correction are Then the coordinates of the highest point after pitch correction are... for: ; Simultaneously, as the gimbal rotates, it will produce a slight tilt around the z-axis of the coordinate system. If at this time the tilt around the z-axis is counterclockwise... If the degree is given, then the corrected coordinates of the highest point are... for: ; The data correction module (7) performs translation correction, including: Translation describes a point The coordinate system in question underwent a linear shift, moving along the negative coordinate axes by [missing information]. , , The distance, in the new coordinate system The coordinates are obtained by giving Add a translation vector accomplish: ； When the gimbal rotates upwards When the angle is equal to 10 degrees, similar triangles can be used to determine the rotation axis of the gimbal. Degrees; Let Δh be the distance from the optical center of the camera to the axis of rotation of the gimbal. The resulting translation along the Y-axis... for: ； The translation that occurs in the Z-axis direction at this time for: ； Let the coordinates of the highest point in the 3D point cloud at this moment be... The coordinates after translation correction are Then the coordinates after translation correction are: ； The data correction module (7) comprehensively considers pitch correction and translation correction. Let the coordinates of the highest point in the three-dimensional point cloud before correction be... The corrected coordinates are Then the corrected coordinates of the highest point are... for: .

2. The system for omnidirectional panoramic measurement of ship height based on a gimbal and binocular camera (1) according to claim 1, characterized in that, Two binocular cameras (1) are placed horizontally on a gimbal, and the tilt sensor (2) is placed near the left camera.

3. The system for omnidirectional panoramic measurement of ship height based on a gimbal and binocular camera (1) according to claim 1, characterized in that, The camera calibration module (3) adopts the Zhang Zhengyou calibration method. It takes pictures of the checkerboard pattern at different positions and uses the calibration algorithm to solve the camera parameters to obtain the camera's intrinsic parameter matrix and distortion coefficient, as well as the extrinsic parameter matrix between the left and right cameras.

4. A method for omnidirectional panoramic measurement of ship height based on a gimbal and binocular camera, characterized in that, The method includes: S1. Perform internal and external parameter calibration on the stereo camera, and calculate the intrinsic parameter matrix, distortion coefficient, and extrinsic parameter matrix of the left and right cameras respectively. S2. Control the gimbal to rotate left and right so that the ship appears in the field of view of the binocular camera; S3. Correct the image acquired by the camera at this time according to the camera's intrinsic parameters; S4. Detect the highest point through the highest point detection module, and adjust the pitch and lateral of the control gimbal based on the highest point to make the highest point of the ship appear in the field of view and obtain the coordinates of the highest point, while tracking the highest point of the ship. S5. Based on the intrinsic and extrinsic parameters of the binocular camera, perform binocular stereo matching on the images acquired by the left and right cameras, and generate a disparity map and corresponding 3D point cloud with the left camera as the reference. S6. Perform pitch correction on the generated 3D point cloud based on the current pitch angle of the binocular camera; S7. Perform translation correction on the generated 3D point cloud based on the current pitch angle of the binocular camera and the radius of the gimbal rotation axis. S8. Obtain the true height of the ship based on the corrected 3D point cloud of the ship's highest point; The highest point detection module detects the highest point by: after calibrating the camera's intrinsic parameters, taking the image of the left camera after intrinsic parameter correction, performing background modeling on it, extracting the moving ship target, and performing binarization processing on the image, setting the pixel where the ship target is located to 1 and the rest to 0; Then, an opening operation is performed on the image to remove isolated noise points; Traverse the pixels from bottom to top. If a row of pixels is all 0, then the row below that row contains the highest point, and retrieve the coordinates of the highest point. If no row of pixels is all 0, control the gimbal to rotate upwards until a row of pixels is detected to be all 0. The pitch correction includes: Camera coordinate system rotation x If the axis rotates counterclockwise, then at this time... x Axis rotation Let the coordinates of the highest point in the 3D point cloud be (degrees). The coordinates after pitch correction are Then the coordinates of the highest point after pitch correction are... for: ; Simultaneously, as the gimbal rotates, it will produce a slight tilt around the z-axis of the coordinate system. If at this time the tilt around the z-axis is counterclockwise... If the degree is given, then the corrected coordinates of the highest point are... for: ; The translation correction includes: Translation describes a point The coordinate system in question underwent a linear shift, moving along the negative coordinate axes by [missing information]. , , The distance, in the new coordinate system The coordinates are obtained by giving Add a translation vector accomplish: ； When the gimbal rotates upwards When the angle is equal to 10 degrees, similar triangles can be used to determine the rotation axis of the gimbal. Degrees; Let Δh be the distance from the optical center of the camera to the axis of rotation of the gimbal. The resulting translation along the Y-axis... for: ； The translation that occurs in the Z-axis direction at this time for: ； Let the coordinates of the highest point in the 3D point cloud at this moment be... The coordinates after translation correction are Then the coordinates after translation correction are: ； Taking into account both pitch and translation corrections, let the coordinates of the highest point in the 3D point cloud before correction be... The corrected coordinates are Then the corrected coordinates of the highest point are... for: .

Images