A multi-view camera ship height measurement method based on dynamic long baseline

Abstract

The application discloses a kind of multi-view camera ship height measurement methods based on dynamic long baseline, comprising: S1: river is divided into three equal-width regions;S2: two cameras of different baseline values in three side-by-side cameras are formed into binocular shooting system, and shooting is carried out in three equal-width regions, and three different left view images and right view images are obtained;S3: the pre-calibration parameter threshold range of each river binocular shooting system is obtained;S4: the measured distance is compared with the pre-calibration parameter threshold range, so that the combination of shooting system is dynamically adjusted;S5: in multi-view camera system, the shooting system combined by two cameras corresponding to the distance value under different baseline distances is dynamically adjusted, and image data is obtained by shooting the ship sailing;S6: the image data obtained by binocular shooting system is binarized, pixel point is matched, disparity value and all pixel point disparity map are solved, and the distance ship height is obtained in combination with the geometric relationship between depth of field and field of view angle.

Description

Technical Field

[0001] This invention relates to the field of ship height measurement, and in particular discloses a method for measuring ship height using a multi-view camera based on a dynamic long baseline. Background Technology

[0002] With the rapid development of inland shipping in my country, the volume of ships passing through various dam areas has been increasing year by year. While the shipping industry is developing rapidly, the number of newly built bridges on waterways is also increasing. However, the construction of navigation locks in dam areas has not taken into account future shipping conditions. As a result, some large ships often collide with the navigation locks in dam areas when they pass through because their height exceeds the maximum limit. This not only seriously endangers the shipping safety of the ships themselves, but also poses incalculable dangers to surrounding personnel and crew members, causing enormous harm.

[0003] However, the development of ship height measurement technology in China is relatively lagging. Most of the existing mature ship height measurement technologies focus on laser-based ultra-high altitude detection schemes. However, the core of this method is over-altitude warning, not actual ship height measurement. Furthermore, the installation and maintenance costs of laser equipment are high, and operating in relatively humid environments around shipping areas accelerates equipment aging. The application results are not ideal. Another common method is stereoscopic vision height measurement using binocular cameras. This method uses a camera to replace the human eye, obtaining a parallax distance approximating the human eye through camera parameter calibration, and combining this with image matching algorithms to obtain a two-dimensional planar imaging coordinate system derived from real-world coordinates, thereby measuring the ship's height. However, this traditional solution has certain drawbacks. First, after the camera's own parameters are pre-calibrated, the camera's relative position cannot be moved. At this time, when measuring the ship's height by simulating the distance between the ship and the camera during the calibration of the binocular camera's own parameters, accurate height data can be obtained. However, using a fixed set of calibration data to measure the height information of ships at different distances will cause the pre-calibration parameters of the binocular camera, which has a fixed relative position, to not be adjusted in real time to adapt to the distance between the camera and the ship. This will cause the measured height data of the ship to deviate and cannot be corrected. Summary of the Invention

[0004] To address the problems existing in the prior art, this invention discloses a multi-view camera-based method for measuring ship altitude based on dynamic long baselines, comprising the following steps:

[0005] S1: Based on the degree of deviation of the vessel from the lock during its journey in the river, the river channel is divided into three areas of equal width;

[0006] S2: Based on the distance between the divided equal-width regions, two cameras with different baseline values ​​among the three side-by-side cameras are combined into a binocular shooting system to take pictures in the three equal-width regions and obtain three different left view images and right view images.

[0007] S3: Obtain the threshold range of pre-calibration parameters for the binocular imaging system in each river channel and the threshold range of the distance between the ship and the imaging system under the corresponding baseline value;

[0008] S4: The eye-tracking camera system takes pictures and measures the sailing vessel, compares the measured distance data with the pre-calibrated distance threshold range, and then dynamically adjusts the combination of the shooting system.

[0009] S5: A shooting system composed of two cameras that dynamically adjust to match the distance value at different baseline distances in a multi-camera system, to capture images of sailing ships, and to preprocess the image data.

[0010] S6: The image data obtained by the binocular shooting system is binarized and pixel matching is performed to obtain the disparity value and the disparity map of all pixels. The distance to the ship height is obtained by combining the geometric relationship between depth of field and field of view.

[0011] Furthermore, the pre-calibration parameter threshold range is obtained as follows: the left and right view images are transformed from the world coordinate system to the camera coordinate system through rigid body transformation, then transformed from the camera coordinate system to the image coordinate system, and finally transformed twice to obtain the pixel coordinates in the two-dimensional plane; the transformation matrix between the pixel coordinate system and the image coordinate system is multiplied by the transformation matrix between the camera coordinate system to obtain the intrinsic parameter matrix of the stereo camera shooting system, and the matrix of transformation between the world coordinate system and the camera coordinate system is the extrinsic parameter matrix of the stereo camera shooting system. The corresponding pre-calibration parameter threshold range of the stereo camera shooting system is obtained based on the intrinsic and extrinsic parameter matrices.

[0012] Furthermore, when dynamically adjusting the combination of the shooting systems: when a passing ship passes by, the binocular camera shooting system, consisting of the camera that first captures the ship and the nearby camera, first captures and measures the data. The captured data is compared with the pre-calibrated parameters. If the measured ship data is not within the threshold range of the pre-calibrated parameters, the multi-camera system automatically switches to the binocular camera shooting system under different baseline values ​​for shooting and measurement. If the measured data meets the threshold range of the pre-calibrated parameters, the left and right view images are subjected to epipolar constraints and algorithm processing to correct the image data captured by the left and right cameras and obtain a corrected image.

[0013] Furthermore, for each row of pixels in the corrected image, corresponding pixel matching is performed, and the disparity value formed by the corresponding point in the left and right images is calculated. The disparity value is the column coordinate of the matching point of the pixel in the right camera image minus the column coordinate of the matching point of the corresponding point in the left camera image. The disparity value is calculated for all pixels, and finally a disparity map composed of all pixels is obtained, and the disparity map is optimized.

[0014] Furthermore, each frame of ship image captured by each camera in the binocular imaging system is processed into grayscale binarization and compared with the previous frame image captured by the camera to obtain the outline region of the moving ship. The highest point of the moving region is traversed to obtain the coordinate data of the highest point.

[0015] Furthermore, based on the depth of field, field of view, and spatial geometric relationships, the horizontal distance between the binocular camera and the ship is first calculated. The coordinates are then reversed from the two-dimensional pixel coordinate system to the three-dimensional world coordinate system, and the two-dimensional image data is converted into three-dimensional real-world image data. Finally, the actual height of the ship is obtained. The distance from the ship to the camera is calculated based on the principle of triangulation, and the horizontal distance coordinates are obtained. The disparity map, the pre-calibrated parameter threshold range, and the two-dimensional pixel coordinates of the ship's outline area are combined to reproject the two-dimensional point coordinates in the image into three-dimensional real-world coordinates. The coordinate data of the highest point are then substituted into the reprojected real-world coordinates to obtain the actual height and distance values ​​of the ship.

[0016] By employing the above-mentioned technical solution, this invention provides a multi-camera method for measuring ship height based on dynamic long baselines. This method utilizes a multi-camera system to achieve full coverage of the river channel by dividing it into three equally divided river areas. The three cameras of the multi-camera system are paired up to form a binocular imaging system with appropriate baseline values ​​to capture images and obtain pre-calibrated parameter threshold ranges. This provides a standard measurement basis for subsequent ship navigation measurements. When a ship passes by, the camera first captured by one of the three side-by-side cameras, along with its neighboring camera, forms a binocular imaging system to measure data. If the distance data measured under this baseline value does not conform to the pre-obtained pre-calibrated parameter threshold range, the system automatically switches to a binocular imaging system with different baseline values ​​to re-capture and measure. This process is repeated until a binocular imaging system with appropriate baseline values ​​is obtained. At this point, the imaging system acquires image data, which is then processed by subsequent algorithms and displayed on a computer terminal as the actual ship height and distance value. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating the principle of dividing river channel distances.

[0019] Figure 2 This is a diagram showing the transformation between the world coordinate system and the camera coordinate system during the calibration of a binocular camera.

[0020] Figure 3 This is a diagram showing the transformation between the pixel coordinate system and the image coordinate system during the calibration of a binocular camera.

[0021] Figure 4 This is a diagram showing the transformation from the camera coordinate system to the image coordinate system during the calibration of a binocular camera.

[0022] Figure 5 This is a flowchart of the process for measuring ship altitude using a multi-camera system.

[0023] Figure 6 This is a diagram showing the transformation relationship between the three spatial components from the world coordinate system to the camera coordinate system during binocular camera calibration.

[0024] Figure 7 Parallax diagram illustrating the imaging principle of a binocular camera.

[0025] Figure 8 A diagram showing the relationship between the 3D coordinates in the camera coordinate system and the imaging in the image coordinate system.

[0026] Figure 9 This is a schematic diagram of multi-view camera installation and measurement.

[0027] Figure 10 This is a schematic diagram of the rotation matrix during binocular camera calibration. Detailed Implementation

[0028] To make the technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention:

[0029] like Figure 1 The method for measuring ship altitude using a multi-view camera based on a dynamic long baseline, as shown, specifically includes the following steps:

[0030] S1: Based on the degree of deviation of the vessel from the lock during its journey in the river, the river channel is divided into three areas of equal width;

[0031] S11: Based on the degree of deviation of a vessel from the dam gate when it sails from a distance, the entire river channel is divided into three equal-width river-distance zones.

[0032] S12: Based on the distance between the different river areas and the binocular camera shooting points obtained from the division, shooting and measurement are carried out at the farthest distance position, the middle distance position, and the closest distance position from the measurement point in these three divided areas.

[0033] S2: Based on the distance of the divided equal-width regions, two cameras with different baseline values ​​among the three side-by-side cameras are combined into a binocular shooting system to take pictures in the three equal-width regions and obtain three different left-view and right-view images.

[0034] S3: Obtain the threshold range of pre-calibration parameters for the binocular imaging system in each river channel and the threshold range of the distance between the ship and the imaging system at the corresponding baseline value.

[0035] S31: Based on the farthest, middle and nearest distances of the distance measurement points divided into three equal-width river areas, any two cameras in the multi-camera system are paired up to capture multiple sets of left and right camera image data.

[0036] S32: Perform coordinate transformation on the three sets of captured image data, using the world coordinate system ((X... W Y W Z W The camera coordinate system (X) is obtained through rigid body transformation. c Y c Z c ),

[0037]

[0038] Then transform from the camera coordinate system to the image coordinate system (x,y).

[0039]

[0040] Finally, the image coordinate system undergoes a second transformation to obtain the pixel coordinate data (u, v) in a two-dimensional plane.

[0041]

[0042] Represented in matrix form as follows:

[0043]

[0044] S33: The matrix obtained by multiplying the transformation matrix between the pixel coordinate system and the image coordinate system with the transformation matrix between the camera coordinate system is the intrinsic parameter matrix of the stereo camera shooting system.

[0045]

[0046] The transformation matrix between the world coordinate system and the camera coordinate system is the extrinsic parameter matrix of the stereo camera system, which is the rotation and translation matrix required for the two coordinate system transformation. The rotation matrix is ​​as follows:

[0047]

[0048] Because it is controlled by components in the x, y, and z directions, it has three degrees of freedom, and we can define them separately.

[0049] Rotate it around the x-axis, y-axis, and z-axis to obtain the rotation matrices R1, R2, R3 for its respective parts.

[0050] Rotating about the x-axis yields R1:

[0051] Rotating about the y-axis yields R2:

[0052] Rotating about the z-axis yields R3:

[0053] The translation matrix T is

[0054]

[0055] Therefore, the rotation matrix R = R1R2R3 is a 3x3 matrix, and the translation matrix T is a 3x1 matrix. Thus, the extrinsic parameter matrix after camera calibration is:

[0056]

[0057] The coordinates (X, X) of a point P in a 3D scene in the world coordinate system W Y W Z W The relationship between the coordinates (u, v) of its projection point Pu on the camera's imaging plane and the coordinates of Pu is as follows:

[0058]

[0059] Where M is the projection matrix, f is the focal length of the camera, and f x f is the focal length of the camera in the x-direction. y Let f be the focal length of the camera in the y-direction, (u0, v0) be the principal point of the camera, and M1 be completely determined by f. x f y The parameters of the camera are determined by u0 and v0, and are called the camera's intrinsic parameters; M2 is determined by the rotation matrix and translation vector t, and is called the camera's extrinsic parameters. w These are the homogeneous coordinates of a point in the world coordinate system. This allows us to obtain the corresponding pre-calibration parameter threshold range for the binocular camera system.

[0060] S34: The baseline distances of the binocular imaging systems corresponding to the three different distance measurement points in the river are different. For river channels with different distances, the pre-calibrated distance threshold range from the binocular imaging system to the photographed vessel under the corresponding baseline value is determined.

[0061] S35: The three critical distance points of the farthest, nearest and middle of the three rivers are calibrated to obtain three sets of pre-calibration parameter threshold ranges.

[0062] S36: The threshold ranges of the three sets of pre-calibrated parameters, the distance threshold range, and the corresponding baseline values ​​are matched one-to-one and used as the basis for future measurement judgments.

[0063] A binocular imaging system is used to measure the distance to a ship in transit. The measured distance data is compared with the threshold range of pre-calibrated parameters, thereby dynamically adjusting the combination of the imaging system.

[0064] When a vessel passes through the S41 gate, the multi-view camera system, consisting of the camera that first captures the image of the vessel based on its direction of travel (either towards or away from the gate) and a nearby camera, takes the first image for measurement.

[0065] S42 The ship distance data captured and measured by the imaging system at the baseline distance is compared with the ship distance threshold range pre-measured at the baseline distance. If the measured ship data is less than or exceeds the threshold range, it means that the binocular system at this baseline cannot accurately measure the ship's height information. The multi-camera system automatically switches between two cameras at different baseline values ​​to form a binocular camera imaging system for imaging and measurement, and repeats the analysis until an accurate camera imaging combination is obtained.

[0066] S5: A shooting system composed of two cameras that dynamically adjust to match the distance value at different baseline distances in a multi-camera system, to capture images of sailing ships, and to preprocess the image data.

[0067] S51: If the measurement data meets the pre-calibrated distance threshold range, the binocular imaging system at this baseline value acquires data for the left and right view images.

[0068] S52: Perform epipolar constraints on the acquired left and right view image data. Starting from the first point in the upper left corner of the image, traverse from left to right and from top to bottom to make the left and right images lie on the same plane and are parallel to each other, thus completing the correction and realizing left and right imaging.

[0069] S6: The image data obtained by the binocular shooting system is binarized and pixel matching is performed to obtain the disparity value and the disparity map of all pixels. The distance to the ship height is obtained by combining the geometric relationship between depth of field and field of view.

[0070] S61: For each row of pixels in the left and right imaging, perform pixel matching for the corresponding target point. The distance between the lines connecting the projection centers of the two cameras is b. The imaging point of any point P in 3D space on the left camera is P_i. L The imaging point of the right camera is P. R According to the principle of rectilinear propagation of light, point P in three-dimensional space is the intersection of the lines connecting the projection centers of the two cameras and the image point. Line segment P L and P R Let P be the distance from the image point of the left and right cameras to the left image plane, respectively. Then, the parallax of point P on the left and right cameras can be defined as follows:

[0071] d=|x L -x R |

[0072] Therefore, the disparity value formed by the corresponding point in the left and right images can be calculated.

[0073] S62: By calculating the disparity for each pixel, it is possible to obtain the original... Figure 1 A disparity map of similar size indirectly represents the matching relationship of the images. Finally, the obtained disparity map is optimized to obtain the processed disparity data.

[0074] S63: Subtract the current frame image from the pre-stored or real-time acquired background image. Regions with differences exceeding a set threshold are considered moving target regions. This background subtraction algorithm is used to detect moving ship objects and delineate moving target regions. Then, each frame image captured by the camera is converted into a binary image. Moving ship targets have a grayscale value of 1, and the image is displayed as white. Static background regions have a grayscale value of 0, and the image is displayed as black. The binary image is iterated through each pixel sequentially, and the first pixel with a value of 1 is considered the highest point in the ship image.

[0075] S64: The disparity map, combined with pre-calibrated parameter thresholds and the two-dimensional pixel coordinates of the ship's outline region, reverse-engineers the three-dimensional world coordinate system coordinates from the two-dimensional pixel coordinate system, realizing the conversion from two-dimensional image data to three-dimensional reality image data.

[0076] S65: Based on the geometric relationships between depth of field, field of view, and space, and using the principle of triangulation, two imaging points P are used. L and P R The distance between them is:

[0077]

[0078] According to the theory of similar triangles, we can conclude that:

[0079]

[0080] Then the distance Z from point P to the projection center plane can be obtained.

[0081]

[0082] When point P moves in three-dimensional space, its image position on the left and right cameras changes, resulting in a corresponding change in parallax. As shown in the above equation, parallax is inversely proportional to the distance from the point in three-dimensional space to the projection center plane. Therefore, knowing the parallax of a point allows us to determine its depth. Based on the principle of similar triangles, the following relationship holds:

[0083]

[0084] By obtaining the parallax of any point in 3D space across different images, and then determining the 3D coordinates of that point based on the pre-calibrated parameter threshold range of the camera system, the 3D coordinates of that point can be determined. This allows for the calculation of the depth coordinates (Z) and altitude coordinates (Y) in the real world from the coordinates of a 2D pixel. This, in turn, yields the ship's distance and altitude information.

[0085] Example:

[0086] The specific implementation of this method includes the following steps:

[0087] Three identical cameras are mounted on a support along one side of the river channel, with their optical axes parallel to each other. The focal lengths of the three cameras are determined. The distances between the three cameras, from near to far along the three pre-defined river channels, correspond to the baseline distances d1, d2, and d3 (i.e., baseline lengths) between any two cameras forming a binocular system. The field of view of any two cameras in the binocular system overlaps, and the vertical height h of the cameras above the ground is maintained. This ensures that passing ships are fully captured by the binocular cameras for imaging. Simultaneously, when no ships are passing through the river, a water level gauge is installed at the junction of the river channel and the water surface to monitor water level fluctuations in real time and obtain corresponding analytical data. Image coordinate systems, world coordinate systems, pixel coordinate systems, and camera coordinate systems are established. Through camera calibration, the intrinsic and extrinsic parameters of each camera are calculated. Figure 4 As shown, the relationship between the camera coordinate system and the image coordinate system is as follows: triangle DAO C Similar to O i CO C Triangle BAO C Similar to PCO C Furthermore, based on the correspondence between similar triangles, we can know that:

[0088]

[0089] Therefore, it can be deduced that

[0090]

[0091]

[0092]

[0093] By writing it in matrix form, we obtain the correspondence between the camera coordinate system and the image coordinate system:

[0094] The result of the transformation is:

[0095]

[0096] like Figure 3 As shown, assuming the center pixel coordinates of the image are (u0, v0), and the physical size of each pixel in the camera's sensor is dx*dy, then the relationship between the coordinates (x, y) of the image coordinate system and the coordinates (u, v) of the pixel coordinate system can be expressed as:

[0097]

[0098] Written in matrix form:

[0099]

[0100] Rewritten in homogeneous coordinates:

[0101]

[0102] Represented in another matrix form as follows:

[0103]

[0104] In summary, multiplying the two matrices above yields the camera's intrinsic parameter matrix:

[0105]

[0106] Because both the world coordinate system and the camera coordinate system are right-handed coordinate systems, they will not undergo deformation, such as Figure 10 As shown. Solving for the camera's extrinsic parameters requires solving for the rotation and translation matrices needed for the coordinate system transformation. The solution for the rotation matrix is ​​shown below.

[0107]

[0108] The matrix is ​​represented as:

[0109]

[0110] Because it is controlled by components in the x, y, and z directions, it has three degrees of freedom. We obtain the rotation matrices R1, R2, and R3 of its respective components by rotating it around the x, y, and z axes, respectively. Figure 6 a, b, and c in the figure represent:

[0111] Rotating about the x-axis yields R1:

[0112] Rotating about the y-axis yields R2:

[0113] Rotating about the z-axis yields R3:

[0114] And translation matrix

[0115] Therefore, the rotation matrix R = R1R2R3 is a 3*3 matrix, and the translation matrix T is a 3*1 matrix.

[0116] Therefore, the extrinsic parameter matrix after camera calibration is: Combining this with the formula above, we can now obtain the coordinates (X, Y, X) of a point P in the 3D scene in the world coordinate system. W ,Y W Z W The relationship between the coordinates (u,v) of its projection point Pu on the camera's imaging plane and the coordinates of Pu is as follows:

[0117]

[0118] Where M is the projection matrix, f is the focal length of the camera, and f x f is the focal length of the camera in the x-direction. y Let f be the focal length of the camera in the y-direction, (u0, v0) be the principal point of the camera, and M1 be completely determined by f. x f y The parameters of the camera are determined by u0 and v0, and are called the camera's intrinsic parameters; M2 is determined by the rotation matrix and translation vector t, and is called the camera's extrinsic parameters. w Here are the homogeneous coordinates of the spatial point in the world coordinate system. After camera calibration, two cameras from the multi-view camera system are used to capture images of the ship, thus collecting ship information. To collect ship information, the system should activate the two pre-calibrated cameras on the support. This requires us to pre-divide the ship's distance into three threshold intervals, such as... Figure 1As shown, based on these three width distances, the accurate calibration parameters of the binocular system formed by any two cameras in the multi-camera system on the support and their baseline distances at these three width distances are determined for each distance. A threshold range for accurately measuring ship distance information obtained from these accurate parameters is also determined. The above steps are repeated multiple times, that is, the critical values ​​of the farthest, middle, and closest distances at these three width distances are calibrated using a binocular system formed by two cameras at different baselines that meet the requirements. At this point, the accurate calibration parameters of the cameras that meet the pre-defined three distance requirements and the camera calibration parameters corresponding to the farthest distance at each of the three width distances are obtained. Thus, the threshold range of ship distance at this baseline is measured, and subsequent judgments are made based on this threshold range. To determine whether the binocular system under this baseline can accurately measure ship height data, when a ship traveling in the river passes the elevation measurement point and heads towards or leaves the lock, the multi-camera system is activated. It compares the measurement results of one side of the binocular camera system with the pre-measured ship distance threshold range under the baseline of that binocular system to determine whether accurate ship data can be measured at that baseline distance. If the measurement result is less than or exceeds the threshold range, it means that the binocular system under that baseline distance cannot accurately measure the ship height data. Therefore, the multi-camera system will automatically switch to cameras under different baselines to re-form a binocular system, that is, activate another set of binocular camera systems under another baseline for measurement, and so on, until accurate ship height data is finally obtained to achieve the purpose of dynamic long baseline measurement of ship height.

[0119] Construct a ship height calculation module

[0120] Through the measurement module, we obtain the basic outline information of the ship, acquiring the outline and the coordinates of the pixels in the two-dimensional image within the outline. Simultaneously, we apply epipolar constraints to the left and right ship images obtained from the binocular camera preprocessing, ensuring that the two images are on the same plane and parallel to each other. Specifically, starting from the first point in the upper left corner of the image, we traverse from left to right and from top to bottom to correct the left and right ship images. Furthermore, for each row of pixels in the left and right images, we perform pixel matching for the corresponding target point, and finally calculate the disparity formed by the target point in the two images. The disparity obtained from the binocular camera images at the baseline distance can be used to calculate the similarity between the point to be matched and the point to be matched using a semi-global stereo matching algorithm. The higher the similarity between two points, the greater the probability that they are the corresponding matching points. To address the issue of the computational scope being limited to a local window in cost matching, a global energy optimization strategy is adopted. By setting a global energy function and continuously optimizing it, the function's objective function is minimized, ensuring that each pixel is the optimal match. Finally, for the disparity calculation, the optimal disparity represented by each pixel is the minimum cost aggregation value. By calculating the time difference for each pixel, the optimal disparity can be obtained from the original... Figure 1 A disparity map of the same size indirectly represents the matching relationship of the images. Finally, optimizing the obtained disparity map yields processed disparity data, enabling the computer to perceive the three-dimensional world. This allows us to obtain the depth and height coordinates in the real world from the coordinates of two-dimensional pixels. Figure 7 The distance between the lines connecting the projection centers of the two cameras shown is b. The image point of any point P in 3D space on the left camera is P_i. L The imaging point of the right camera is P. R According to the principle of rectilinear propagation of light, point P in three-dimensional space is the intersection of the lines connecting the projection centers of the two cameras and the image point. Line segment P L and P R Let P be the distance from the image point of the left and right cameras to the left image plane, respectively. Then, the parallax of point P in the left and right cameras can be defined as follows:

[0121] d=|x L -x R |

[0122] The distance between the two imaging points PL and PR is:

[0123]

[0124] According to the theory of similar triangles, we can conclude that:

[0125]

[0126] Then the distance Z from point P to the projection center plane can be obtained.

[0127]

[0128] As point P moves in 3D space, its image position on the left and right cameras changes, resulting in a corresponding change in parallax. As shown in the above equation, parallax is inversely proportional to the distance from the point in 3D space to the projection center plane. Therefore, knowing the parallax of a point allows us to determine its depth information. The image of point P on the camera in 3D space is formed by... Figure 8 As can be seen from the principle of similar triangles, the following relationship exists:

[0129]

[0130] Therefore, given the parallax of any point in three-dimensional space across different images, and based on camera parameters, the three-dimensional coordinates of that point can be determined. This enables computers to perceive the three-dimensional world, allowing us to derive the depth coordinates (Z) and height coordinates (Y) in the real world from the coordinates of two-dimensional pixels. This provides the ship's distance and height information. This method overcomes the limitations of traditional binocular camera systems, which, after obtaining a fixed distance threshold for pre-calibrated parameters, lack flexibility when measuring ship height data at different distances. Measurements of ships exceeding or falling below this fixed distance range are inaccurate, resulting in uncorrectable height deviations. This method utilizes a multi-camera system—three cameras forming a single imaging system—to achieve full coverage of the river channel. By combining different baseline values ​​within the multi-camera system, the binocular imaging system accurately measures ships at different distances, preventing significant deviations in the captured data that do not reflect reality. Meanwhile, the measurement method operates automatically throughout the entire process, driven by algorithms to calculate and measure, saving labor costs and preventing deviations in measurement data due to human factors. It is also unaffected by environmental factors and can provide timely and efficient early warnings during ship navigation, effectively protecting people's lives and navigation safety.

[0131] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A multi-view camera ship height measurement method based on dynamic long baseline, characterized by The method comprises the following steps: S1: According to the deviation of the ship from the lock in the river, the river is divided into three equal width regions; S2: According to the distance of the divided equal width region, two cameras with different baseline values in the three parallel cameras form a binocular camera system, and the three equal width regions are photographed to obtain three different left view images and right view images; S3: Obtain the pre-calibration parameter threshold range of the binocular camera system under each river, and the distance threshold range between the ship and the camera system under the corresponding baseline value; S4: The multi-camera system measures the ship, compares the measured distance data with the pre-calibration distance threshold range, and dynamically adjusts the combination of the camera system; S5: In the multi-camera system, the two cameras corresponding to the distance value are combined to form a camera system, the ship is photographed to obtain image data, and the image data is preprocessed; S6: The image data obtained by the binocular camera system is binarized and pixel point matched to obtain the disparity value and all pixel point disparity maps, and the ship distance and height information are obtained by combining the geometric relationship between the depth of field and the field of view angle; When the combination of the camera system is dynamically adjusted: when the ship passes, the multi-camera first measures the ship by the binocular camera system composed of the camera that first photographs the ship in the running direction of the ship towards or away from the lock; The ship distance data measured by the camera system under the baseline distance is compared with the pre-measured ship distance threshold range under the baseline distance. If the measured ship data is less than or exceeds the threshold range, it means that the binocular system under this baseline cannot accurately measure the ship height information, and the multi-camera system automatically switches the two cameras under different baseline values to form a binocular camera system for measurement, and the analysis is repeated until the accurate camera combination is obtained.

2. The method of claim 1, wherein: The pre-calibration parameter threshold range is obtained by the following method: the left view image and the right view image are converted from the world coordinate system to the camera coordinate system by rigid transformation, and then converted from the camera coordinate system to the image coordinate system, and finally converted to pixel coordinates in the two-dimensional plane; The conversion matrix between the pixel coordinate system and the image coordinate system is multiplied by the conversion matrix between the camera coordinate system to obtain the intrinsic matrix of the binocular camera system. The matrix between the world coordinate system and the camera coordinate system is the extrinsic matrix of the binocular camera system. According to the intrinsic and extrinsic parameter matrices, the pre-calibration parameter threshold range of the corresponding binocular camera system is obtained.

3. The method of claim 2, wherein: Each row of pixel points on the corrected image is matched with the corresponding pixel points, the disparity value formed by the corresponding points on the left and right images is calculated, the disparity value is the column coordinate of the matching point of the pixel point on the right camera image minus the column coordinate of the matching point of the corresponding point on the left camera image, the disparity value of all pixel points is calculated, and finally the disparity map composed of all pixel points is obtained, and the disparity map is optimized.

4. The method of claim 1, wherein: The ship image of each frame photographed by each group of cameras in the binocular shooting system is subjected to gray scale binary processing and compared with the previous frame image photographed by the camera to obtain a moving ship contour region, and the highest point coordinates data of the moving region are obtained by traversing the highest point.

5. The method of claim 4, wherein: According to the geometric relationship between the depth of field, the field of view angle and the space, the horizontal distance between the binocular camera and the ship is solved, the three-dimensional world coordinate system is inversely deduced from the two-dimensional pixel coordinate system, the conversion from the two-dimensional image data to the three-dimensional real image data is performed, the actual height of the ship is finally obtained, the distance from the ship to the camera is calculated based on the principle of triangulation, the horizontal distance coordinates are obtained, the parallax map, the pre-calibration parameter threshold range and the two-dimensional pixel point coordinates of the ship contour region are combined, the two-dimensional point coordinates in the image are re-projected into the three-dimensional real world coordinates, the highest point coordinate data is brought into the re-projected real world coordinates, and the actual height and distance value of the ship are obtained.

Images