Automatic measurement method for inland river ship freeboard based on monocular vision

Abstract

The application discloses a kind of inland river ship freeboard measurement method based on monocular vision.First, the internal parameter calibration of monocular camera is carried out using checkerboard by Zhang Zhengyou calibration method, and the internal parameter of camera is obtained;The camera shoots the side view of the ship, and the Deeplabv3+ model is used to segment the part between the ship deck line and the waterline according to the side view of the ship;Further, the ship deck line and the waterline are detected to determine the position of the ship freeboard and calculate the pixel height of the ship freeboard;The conversion from pixel height to actual height is realized by the camera internal parameter obtained by calibration, and the real value of the ship freeboard is calculated.The application models the parameters of the camera itself, does not rely on vanishing point and vanishing line for ship freeboard measurement, and provides assistance for the safety navigation of inland river ships on water as the basis for ship overload detection.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of monocular vision measurement, and relates to ship safety navigation monitoring technology, and realizes a method for automatically measuring the freeboard of an inland river ship based on monocular vision, thereby providing help for the safe navigation of the inland river ship on water. BACKGROUND

[0002] Ship overloading is a major problem in ship navigation and one of the main causes of ship accidents. When a ship is overloaded and navigates, the draft of the ship increases, and the freeboard of the ship is insufficient, which poses a great safety hazard in the event of an emergency, leading to inland river ship collision accidents. By measuring the freeboard of the ship during navigation and comparing it with the minimum freeboard of the ship, it can be determined whether the ship is overloaded.

[0003] The existing method is a ship freeboard measurement based on laser or binocular vision. However, these methods have the following problems: (1) Laser measurement is greatly affected by the environment, is easily disturbed by the outside world, and is relatively complex to install. (2) Binocular vision measurement has high computational complexity and is greatly affected by depth, and the measurement accuracy decreases as the depth increases. There is currently no complete ship freeboard measurement method based on monocular vision.

[0004] In order to fill the gap of the monocular vision method, the present application uses monocular vision to measure the freeboard of the ship. The internal parameters of the monocular camera are modeled, the internal parameters of the camera are obtained through camera calibration, the part between the deck line and the waterline of the ship is segmented through semantic segmentation, the pixel freeboard of the ship is calculated, the pixel freeboard of the ship is converted into the actual freeboard of the ship, and the freeboard measurement of the ship is realized. SUMMARY

[0005] The problem solved by the present application is to address the gap in the existing research of ship freeboard measurement based on monocular vision, and to provide an automatic measurement method for the freeboard of an inland river ship based on monocular vision. The present application is based on the internal parameter calibration and semantic segmentation of a monocular camera. On the basis of semantic segmentation, the picture is divided into grids, the deck line and waterline of the ship are detected, and the freeboard of the ship is measured. The freeboard of the ship is automatically and effectively measured, the freeboard of the ship is measured in real time, it is monitored whether the ship is overloaded during navigation, and the safe navigation of the ship is realized.

[0006] The technical solution of the present application mainly includes the following steps:

[0007] S1: Calibrate the monocular camera to obtain the internal parameters (f x , f y , u0, v0) of the camera;

[0008] S2: According to the monocular camera imaging principle, that is, the pinhole imaging model, the conversion from pixel height to actual height is realized by using the camera internal parameters in step S1;

[0009] S3: By collecting ship picture data, the ship picture data is labeled, the semantic segmentation model is trained, and the segmentation of the ship deck line and the waterline part is realized;

[0010] S4: Using the segmentation result obtained in step S3, the deck line and waterline detection are carried out, and the ship freeboard position is determined, and the ship freeboard pixel height is calculated;

[0011] S5: The above steps are built into a ship freeboard measurement system to realize automatic measurement of ship freeboard and experimental verification.

[0012] The specific implementation method of step S1 is as follows:

[0013] S1-1: The camera imaging process is determined by four coordinate systems, that is, the world coordinate system (X w , Y w , Z w ), the camera coordinate system (X c , Y c , Z c ), the image coordinate system (x, y) and the pixel coordinate system (u, v). The conversion formula from the world coordinate system to the camera coordinate system is:

[0014]

[0015] Where R 3×3 represents a 3x3 rotation matrix, and T 3×1 represents a 3x1 translation matrix. It is used to convert the world coordinate system to the camera coordinate system. The conversion formula from the camera coordinate system to the image coordinate system is:

[0016]

[0017] Where f is the focal length of the camera, z is the depth from the camera coordinate system to the image coordinate system, and the conversion formula from the image coordinate system to the pixel coordinate system is:

[0018]

[0019] Where (u0, v0) is the principal point coordinate of the camera, and dx and dy represent the actual size of the pixels on the photosensitive chip, with the unit of millimeters per pixel. Through the above formula conversion, the conversion formula from the world coordinate system to the pixel coordinate system is:

[0020]

[0021] Where M inis the intrinsic parameter matrix of the camera, where f x , f y are the focal length of the camera in x and y direction, M out is the extrinsic parameter matrix of the camera. The intrinsic parameter matrix of the camera is a fixed property of the camera and does not change with the position of the camera; while the extrinsic parameter matrix of the camera changes with the position of the camera.

[0022] S1-2: Based on the conversion relationship from the world coordinate system to the pixel coordinate system given in step S1-1, assume that the checkerboard calibration board is located at Z c = 0, define the i-th column of the rotation matrix R as r i , then the conversion relationship is as follows:

[0023]

[0024] Let H = M in [r1 r2 t] = [h1 h2 h3], H is a homography matrix, which can be solved by detecting the relationship from the world coordinate system to the pixel coordinate system of the corner points. It is solved by the following formula:

[0025] M in -1 [h1 h2 h3] = λ[r1 r2 t]

[0026] r1 T r2 = 0

[0027]

[0028] At this time, H is the mixed matrix of the intrinsic matrix and the extrinsic matrix obtained after solving, where λ is an arbitrary scalar. Define B = M in -T M in -1 , at this time B is a symmetric matrix, so it has six unknowns, denoted by b, and its formula is:

[0029] b = [B 11 B 12 B 22 B 13 B 23 B 33 ]

[0030] Let the i-th column of H be h i , then h i = [h i1 h i2 h i3 ] T , according to the properties of b, the following formula can be derived:

[0031] hi T Bh j = v ij T b

[0032] v ij = [h i1 h j1 h i1 h j2 + h i2 h j1 h i2 h j2 h i3 h j1 + h i1 h j3 h i3 h j2 + h i2 h j3 h i3 h j3 ] T

[0033] h1 T M in -T M in -1 h2 = 0

[0034] h1 T M in -T M in -1 h1 = h2 T M in -T M in -1 h2

[0035] Using the above formula, the following formula can be derived:

[0036]

[0037] By taking n pictures, the homogeneous equation above can be calculated to obtain a unique solution of b, and then B can be calculated, and through cholesky decomposition, the camera intrinsic parameters can be obtained.

[0038] Further, the step S2 is specifically implemented as follows:

[0039] S2-1: The camera parameters have been obtained through step S1. According to the camera imaging principle, when the camera is imaging, the principal point coordinates of the camera are the intersection of the camera optical axis and the imaging plane, and the camera optical axis is perpendicular to the imaging plane. The distance between the camera center and the imaging plane is the focal length of the camera. In the ideal case, the principal point coordinates of the camera imaging are in the center of the image, but some deviation will occur in the actual assembly process of the camera, so it is necessary to calibrate to obtain accurate camera intrinsic parameters.

[0040] S2-2: According to the invariance of the principal point in the camera imaging process, the projection point of the camera height in the image is calculated, which is used as the prior height to calculate the height of other objects in the scene.

[0041] S2-3: The projection point of the actual height of the camera in the image is calculated, the height of the camera from the reference plane is measured, and the height of the target object is calculated by the following formula. The calculation formula is:

[0042]

[0043] wherein H objp is the pixel height of the object in the image, H camp is the pixel height of the camera to the reference plane, H objs is the real height of the object, and H cams is the real height of the camera to the reference plane. Based on this formula, by measuring the real height of the camera to the reference plane, the conversion from the pixel height to the real height can be realized.

[0044] Further, the step S3 is specifically implemented as follows:

[0045] S3-1: First, the camera collects the ship target data, and uses the annotation software to annotate the part from the ship deck to the water surface.

[0046] S3-2: The collected ship target data is randomly divided into training set and test set according to the corresponding proportion.

[0047] S3-3: The pre-training weight of the semantic segmentation model is used for training to accelerate the convergence of the network. After the training is completed, the optimal weight is reserved.

[0048] S3-4: The collected ship picture is segmented using the trained semantic segmentation model to obtain the segmentation result.

[0049] Further, the step S4 is specifically implemented as follows:

[0050] S4-1: The segmentation result between the ship deck line and the ship waterline obtained in step S3 is extracted to obtain the ship deck line and the waterline, and the ship pixel freeboard is measured.

[0051] S4-2: Because the ship is generally in the state of right floating or longitudinal inclination when sailing, it is almost in the state of transverse inclination, so the transverse inclination state is not considered. When the ship is in the state of right floating, the ship deck line and the water line are parallel, and the distance between the two parallel lines is the ship freeboard. When the ship is in the state of longitudinal inclination, the ship deck line and the water line are not parallel, and the midpoint of the ship is the ship freeboard point, so the modeling analysis of the two scenes needs to be carried out to realize the measurement of the ship freeboard in the two cases.

[0052] S4-3: The midpoint of the deck line is taken as the ship freeboard point O through the deck line and the water line detected in step S4-1, a straight line is made along the vertical direction through the freeboard point to obtain the intersection point P with the water line, and a perpendicular line of the water line is made through the freeboard point to obtain the perpendicular point Z, and the distance between the freeboard point and the perpendicular point is the pixel freeboard of the ship, that is, OZ.

[0053] S4-4: The pixel height of OP is calculated first, and the pixel height is converted into the actual height OP_S through the modeling method of step S2, and then the included angle between OP and OZ is calculated, and the actual freeboard value of the ship is obtained by multiplying OP_S by the cosine value of the included angle.

[0054] Further, the step S5 is specifically implemented as follows:

[0055] In view of the difficulty in measuring the true value of the freeboard of the inland ship in the real scene, the real scene is simulated for experimental verification. A ship model is used as a real-world ship, a cuboid water tank is used as a waterway when the ship is sailing, the real freeboard of the ship model is measured by the algorithm, and the real freeboard of the ship measured manually is compared to verify the reliability of the algorithm.

[0056] The present application has the following advantages:

[0057] 1. The present application provides a simple and effective height measurement method based on monocular vision, which can measure the height of the object in the image by calibrating the internal parameters of the monocular camera, without the need for extracting the vanishing point and the vanishing line by means of the geometric information in the scene, and without the need for changing with the depth of the camera, and the deployment is simple.

[0058] 2. In view of the vacancy of the method for measuring the freeboard of the ship based on monocular vision, the present application provides a scheme for measuring the freeboard of the ship using monocular vision, which detects the ship deck line and the water line through camera calibration and semantic segmentation, divides the picture into grids on the basis of semantic segmentation, and can realize the automatic measurement of the freeboard of the ship when the ship is sailing, without the constraint of geometric environment, and effectively measures the freeboard of the ship.

[0059] 3. The present application models the camera's own parameters, does not rely on vanishing points and vanishing lines for ship freeboard measurement, and provides assistance for inland river ship safe navigation as the basis for ship overload detection. BRIEF DESCRIPTION OF DRAWINGS

[0060] Figure 1 is a method flowchart of the present application;

[0061] Figure 2 is a camera calibration experiment diagram of the present application;

[0062] Figure 3 is a monocular vision scene modeling schematic diagram of the present application;

[0063] Figure 4 is a semantic segmentation model structure diagram of the present application

[0064] Figure 5 is a ship deck line and waterline sampling diagram of the present application

[0065] Figure 6 is a ship pixel freeboard schematic diagram of the present application DETAILED DESCRIPTION

[0066] The technical solutions of the present application will be further described in detail below by combining with the drawings.

[0067] The technical method provided by the embodiments of the present application will be described in detail below by combining with the drawings.

[0068] As shown in the method flowchart of the present application, automatic measurement of ship freeboard is realized, and specifically includes the following steps: Figure 1

[0069] S1: The monocular camera is calibrated by Zhang Zhengyou calibration method using a checkerboard to obtain the internal parameters (f x , f y , u0, v0) of the camera;

[0070] As shown in the checkerboard picture taken for this calibration, the calibration principle and method are as follows: Figure 2

[0071] S1.1: The camera imaging process is determined by four coordinate systems, i.e. the world coordinate system (X w , Y w , Z w ), the camera coordinate system (X c , Y c , Z c ), the image coordinate system (x, y) and the pixel coordinate system (u, v). The conversion formula from the world coordinate system to the camera coordinate system is:

[0072] ​​

[0073] where R 3×3 is a 3x3 rotation matrix, T 3×1 is a 3x1 translation matrix. The formula for converting the world coordinate system to the camera coordinate system is:

[0074]

[0075] where f is the focal length of the camera, and the formula for converting the image coordinate system to the pixel coordinate system is:

[0076]

[0077] where (u0, v0) is the principal point coordinate of the camera, and dx and dy represent the actual size of the pixels on the photosensitive chip, with units of millimeters per pixel. Through the above formula conversion, the conversion formula from the world coordinate system to the pixel coordinate system is:

[0078]

[0079] where M in is the camera intrinsic matrix, where f x , f y are the focal lengths of the image in the x and y directions, and M out is the camera extrinsic matrix. The camera intrinsic matrix is a fixed property of the camera and does not change with the position of the camera, while the camera extrinsic matrix changes with the position of the camera.

[0080] S1.2: The calibration process is to fix the checkerboard calibration board on a plane, use the camera to take pictures of the calibration board from different directions, get the calibration board pictures as shown in Figure 2 , the number of corner points of the calibration board used in this calibration is 88, and the interval between each corner point is 10 mm. Fix the calibration board on the wall and use the camera to take effective pictures for intrinsic parameter calculation. Use the function findChessboardCorners() of opencv to detect the corner points, and then use the function calibrateCamera() to calculate the camera intrinsic parameters.

[0081] S2: According to the principle of monocular camera imaging, i.e. pinhole imaging model, use the camera internal parameters to model, realize the conversion from pixel height to actual height;

[0082] As shown in Figure 3 , the monocular modeling method is shown, and the specific method is described as follows:

[0083] S2.1: The camera internal parameters have been obtained through the S1 step. According to the camera imaging principle, when the camera is imaging, the principal point coordinate of the camera is the intersection of the camera optical axis and the imaging plane, and the camera optical axis is perpendicular to the imaging plane. The distance between the camera center and the imaging plane is the focal length of the camera. In the ideal case, the principal point coordinate of the camera imaging is in the center of the image, but some deviation will occur in the actual assembly process of the camera, so it is necessary to calibrate to obtain accurate camera internal parameters.

[0084] S2.2: According to the invariance of the principal point in the camera imaging process, the projection point of the camera height in the image is calculated, and the height of other objects in the scene is calculated. The formula for calculating the projection point of the camera height in the image is:

[0085] y = v0

[0086] Where v0 is the vertical coordinate of the principal point coordinate of the camera. The projection point of the camera height in the image can be calculated by this formula.

[0087] S2.3: By calculating the projection point of the actual height of the camera on the image, the height of the target object can be calculated by measuring the height of the camera from the reference plane using the following formula. The calculation formula is:

[0088]

[0089] Where H objp is the pixel height of the object in the image, H camp is the pixel height of the camera to the reference plane, H objs is the real height of the object, and H cams is the real height of the camera to the reference plane. Through this formula, the conversion from pixel height to real height can be realized.

[0090] S3: Collect ship picture data, label the ship picture data, and send it into the semantic segmentation model for training to realize the segmentation of the ship deck line and the waterline part;

[0091] As shown in Figure 4 , it is the overall structure diagram of the Deeplabv3+ semantic segmentation model. It uses the encoder Encoder and decoder Decoder structure, where the Encoder includes feature extraction and ASPP hollow space pyramid pooling. The Decoder processes the output of the Encoder to obtain the segmentation result. This structure pays more attention to the boundary information of the object, and improves the segmentation effect. The training method is as follows:

[0092] S3.1: Collect ship data through monocular camera, a total of 1202 pictures of inland river ships, and use the labeling software to label the part from the ship deck to the water surface.

[0093] S3.2: The collected pictures are randomly divided into training set and test set in the ratio of 8:2.

[0094] S3.3: In the last few layers of the feature extraction stage, the convolution is replaced by dilated convolution, which increases the receptive field while maintaining the resolution. The formula is:

[0095]

[0096] where x is the input feature map, y is the output feature map, w is the weight, and the dilated rate r determines the step size used when sampling the input feature map. By adjusting r, the receptive field of the convolution kernel can be changed, allowing the model to learn more information. ASPP empty space pyramid pooling is used to extract features at different scales and upsample them. After feature fusion, a 1x1 convolution is used to fuse features and send them to the Decoder module for decoding.

[0097] S3.4: The loss function used by Deeplabv3+ model is Focal Loss and Dice Loss. Focal Loss is mainly used to solve the imbalance between difficult and easy samples, and its formula is:

[0098]

[0099] where α is a parameter used to balance positive and negative samples, p is the confidence, and γ is used to reduce the loss of high confidence samples. The main purpose of Dice Loss is to solve the strong imbalance between positive and negative samples in semantic segmentation. The Dice coefficient is defined as:

[0100]

[0101] where X is the predicted region, Y is the labeled region, and X∩Y is the intersection of X and Y. Given the Dice coefficient, the Dice Loss is as follows:

[0102]

[0103] Dice Loss is 1 minus the Dice coefficient. By adding the two loss functions together, the model's loss function is obtained, which is used for backpropagation to update the parameters of each layer of the network until the network converges and reaches the optimal solution.

[0104] S3.5: Use the pre-trained weights of Deeplabv3+ on ImageNet dataset to fine-tune the semantic segmentation model on the ship dataset, accelerate the convergence of the network, and keep the best weights after training.

[0105] S3.6: using the trained semantic segmentation model to segment the collected ship picture to obtain the segmentation result.

[0106] S4: using the segmentation result obtained in step S3 to detect the deck line and the waterline, and determining the ship freeboard position and calculating the ship freeboard pixel height; the segmentation result from step S3 separates the target from the background, which facilitates the detection of the deck line and the waterline, and the calculation is as follows:

[0107] S4.1: separating the target from the background through step S3, that is, the image only contains two types of pixel values, the pixel value of the target region and the pixel value of the background region, performing binaryzation processing on the picture, and performing binaryzation operation by setting a threshold, and the formula is:

[0108]

[0109] wherein x is the pixel value, through the binaryzation of the image, the pixel value of the target region is 255, and the pixel value of the background region is 0.

[0110] As shown in Figure 5 , the picture is cut into n parts in proportion to the x axis, the intersection points of the target region and the background region, that is, the points of the upper boundary and the lower boundary of the target region, are found in the n parts, the coordinates of the found points in the image are recorded, and the points are fitted by the least square method to obtain a first-order straight line, and the formula is:

[0111]

[0112] wherein k is the slope of the straight line, x and y are the coordinates of the points, and are the mean values of the x coordinates and the y coordinates of the n points, and n is the number of points, in the case of the known slope k of the straight line, the intercept b can be obtained by the method of undetermined coefficients. The ship deck line and the waterline can be calculated by this method.

[0113] S4.2: because the ship is generally in the state of right floating or longitudinal inclination when sailing, it is almost in the state of transverse inclination, so the transverse inclination state is not considered. When the ship is in the state of right floating, the ship deck line and the waterline are parallel, and the distance between the two parallel lines is the ship freeboard; when the ship is in the state of longitudinal inclination, the ship deck line and the waterline are not parallel, and the midpoint of the ship is the ship freeboard point, so it is necessary to analyze the two scenes to realize the measurement of the ship freeboard in the two cases.

[0114] S4.3: as shown in Figure 6The deck line and the waterline detected by step S4-1 are shown, the midpoint of the deck line is taken as a ship freeboard point O, a straight line is made along the vertical direction through the freeboard point to obtain an intersection point P with the waterline, and a perpendicular line of the waterline is made through the freeboard point to obtain a perpendicular point Z, and the distance between the freeboard point and the perpendicular point is the pixel freeboard of the ship, that is, OZ.

[0115] S4.4: The pixel height of OP is calculated first, the pixel height is converted into an actual height OP_S through the modeling method of S2, and then the included angle between OP and OZ is calculated, and the actual freeboard value of the ship is obtained by multiplying OP_S by the cosine value of the included angle.

[0116] S5: The above steps are built into a ship freeboard measurement system to realize automatic measurement of the ship freeboard, and data is collected for experimental verification, and the verification method is as follows:

[0117] S5.1: In view of the difficulty in measuring the true value of the freeboard of the inland ship in the real scene, a simulation of the real scene is carried out for experimental verification. A ship model is purchased to serve as a real-world ship, a cuboid water tank is purchased to serve as a channel for the ship to sail, the true freeboard of the ship model is measured by the algorithm, and is compared with the manually measured true freeboard of the ship to verify the reliability of the algorithm.

[0118] S5.2: The camera calibrated in S1 is fixed on a camera support, a level is installed on the camera, and the camera is adjusted to be in a horizontal state by adjusting the level. Pictures are taken by the camera and transmitted to a computer, the ship in the pictures is segmented using the trained weights of S3, the deck line and the waterline are detected by S4, the pixel freeboard value of the ship is calculated, the pixel freeboard value of the ship is converted into a true freeboard value using the modeling method of S2, and the true freeboard of the ship is obtained. And compared with the manually measured true freeboard of the ship, the error size is calculated.

[0119] S5.3: The reliability of the method proposed in the application is verified by the following four cases, and the relative error is used to evaluate the results, wherein the formula of the absolute error is:

[0120] Δ = |Y-T|

[0121] Where Y is the ship freeboard value measured by the method proposed in the application, and T is the true value of the ship freeboard. The formula of the relative error is:

[0122]

[0123] Where Δ is the absolute error, and T is the true value of the ship freeboard. The following analyzes the four cases:

[0124] S5.3-1: When the ship is in a positive floating state and the distance between the ship and the camera changes, the ship freeboard is measured.

[0125] S5.3-2: When the ship is in the longitudinal state, and the distance between the ship and the camera changes, the ship freeboard measurement is carried out.

[0126] S5.3-3: When the ship is in the positive floating state, but there is a different angle with the camera, and the distance between the ship and the camera changes, the ship freeboard measurement is carried out.

[0127] S5.3-4: When the ship is in the longitudinal state, but there is a different angle with the camera, and the distance between the ship and the camera changes, the ship freeboard measurement is carried out.

[0128] Through the simulation experiment, the maximum relative error is 9.6%, and through the method proposed by the application, the inland ship freeboard measurement can be effectively carried out, which provides help for the safe navigation of the inland ship.

[0129] The specific embodiments of the application are described above. It should be understood that those skilled in the art of the technical field described in the application can make various modifications or adopt supplements or replace similar methods, but will not deviate from the spirit of the application or exceed the scope defined by the appended claims.

Claims

1. An automatic measurement method for the freeboard of inland waterway vessels based on monocular vision, characterized in that, Includes the following steps: S1: Obtain the camera's intrinsic parameters (f) by calibrating the monocular camera. x f y u0, v0); S2: Based on the imaging principle of a monocular camera, namely the pinhole imaging model, the camera's internal parameters from step S1 are used to model the image and convert the pixel height to the actual height. S3: By collecting ship image data, annotating the ship image data, and training a semantic segmentation model, the segmentation of the ship's deck line and waterline can be achieved; S4: Using the segmentation results obtained in step S3, perform deck line and waterline detection, determine the ship's freeboard position, and calculate the ship's freeboard pixel height; S5: Based on the above steps, build a ship freeboard measurement system to achieve automated measurement of ship freeboard; Using the segmentation results obtained in step S3, the deck line and draft line are detected, including: Step S3 separates the target from the background, meaning the image contains only two types of pixel values: the target region's pixel values ​​and the background region's pixel values. Binarization is then performed on the image by setting a threshold, the formula of which is: Where x is the pixel value, and by binarizing the image, the pixel value of the target area is 255, and the pixel value of the background area is 0; By dividing the image into n equal parts along the x-axis, the intersection points of the target region and the background region (i.e., the points at the upper and lower boundaries of the target region) are found within these n parts. The coordinates of these points in the image are recorded, and a first-order straight line is fitted to these points using the least squares method. The formula is as follows: Where k is the slope of the line, and x and y are the coordinates of the point. and Let x and y be the average of the x and y coordinates of n points, where n is the number of points. Given the slope k of the straight line, the intercept b can be calculated using the method of undetermined coefficients; and then the deck line and waterline of the ship can be calculated.

2. The method for automatic measurement of freeboard of inland waterway vessels based on monocular vision according to claim 1, characterized in that... The specific implementation method of step S2 is as follows: S2-1: Calibration to obtain accurate camera intrinsic parameters; S2-2: Model the camera imaging process based on the principal point invariance, calculate the projection point of the camera height in the image, and use this as the prior height to calculate the height of other objects in the scene. S2-3: By calculating the projection point of the camera's actual height onto the image, and measuring the height of the camera from the reference plane, the height of the target object is calculated using the following formula: Among them, H objp H represents the pixel height of the object in the image. camp H is the pixel height from the camera to the reference plane. objs H represents the actual height of the object. cams This is the true height of the camera from the reference plane; based on this formula, the conversion from pixel height to true height can be achieved by measuring the true height of the camera from the reference plane.

3. The method for automatic measurement of freeboard of inland waterway vessels based on monocular vision according to claim 1 or 2, characterized in that... The specific implementation method of step S3 is as follows: S3-1: First, collect ship target data using a camera, and then use annotation software to annotate the portion of the ship from the deck to the water surface; S3-2: The collected ship target data is randomly divided into training and test sets according to the corresponding proportions; S3-3: Use the pre-trained weights of the semantic segmentation model for training to accelerate network convergence. After training, retain the optimal weights. S3-4: Use the trained semantic segmentation model to segment the collected ship images and obtain the segmentation results.

4. The method for automatic measurement of freeboard of inland waterway vessels based on monocular vision according to claim 3, characterized in that... Step S4, determining the ship's freeboard position, includes: When a ship is upright, its deck line and waterline are parallel, and the distance between these two parallel lines is the ship's freeboard. When a ship is trimmed, its deck line and waterline are not parallel. By measuring the deck line and waterline, the midpoint of the deck line is taken as the ship's freeboard point O. A straight line is drawn vertically through the freeboard point to obtain the intersection point P with the waterline. Then, a perpendicular line is drawn from the freeboard point to the waterline to obtain the perpendicular point Z. The distance between the freeboard point and the perpendicular point is the ship's pixel freeboard, i.e., OZ. First, calculate the pixel height of OP. Then, using the modeling method in step S2, convert its pixel height into the actual height OP_S. Next, calculate the angle between OP and OZ. Multiply OP_S by the cosine of the angle to get the actual freeboard value of the ship.

Images