Improved ship target distance measurement method and system

Abstract

The invention provides an improved ship target distance measurement method and system, and the method comprises the steps: calculating the radial and tangential distortion parameters of a camera and the internal reference of the camera through a camera calibration technology, building a mapping relation through a distortion automatic correction algorithm, carrying out the reverse mapping calculation, carrying out the restoration and filling of vulnerability pixel points, and generating a distortionless correction image. A deep learning model is used to identify a sea level line in a camera video picture, a distortion automatic correction algorithm is used to correct the sea level line into a horizontal straight line, and according to a camera imaging principle, a proportional relation of a tangent function related to a camera lens direction and a sea level angle is constructed. Automatically calculating the installation angle of the camera by using an arc tangent function; and according to the automatically calculated camera installation angle and the pixel point coordinates of the distortionless correction image, the distance between the position of the opposite ship on the corrected sight line video picture and the world coordinate system of the camera is obtained, the ship target distance measurement improvement is realized, and the method has usability and high precision.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent ships, and in particular to an improved ship target ranging method and system. Background Art

[0002] In the field of smart ships, a very important point in ship driving is the safety issue during the ship's operation. That is, during the ship's operation, other ships and the ship need to maintain a safe distance. If the distance is less than this, the crew needs to intervene and perform avoidance operations in advance.

[0003] Although ships have AIS data, from which more accurate distance data from other ships to the ship can be obtained, in actual applications, especially in areas with dense ships near the coast, there are many small boats, and some small boats have a low frequency of AIS signal transmission, so the data validity is poor. Some small boats are not even equipped with AIS equipment. Therefore, it is necessary to use visual algorithms to supplement ship collision avoidance or as a more intuitive display and usage method.

[0004] Existing methods for automated ship surveillance include video ranging algorithms. These algorithms use deep learning to accurately identify targets such as ships. They then use the algorithm to calculate the distance and angle of other ships in the video relative to the ship itself. This data provides a basis for subsequent automated surveillance algorithms. In the near future, this data may even assist with automated driving. However, the practical drawbacks of this video ranging algorithm in the intelligent ship sector are as follows:

[0005] First, the cameras installed on ships tend to capture a wider image, providing the driver with a wider field of view and facilitating their observation work. However, this also results in a larger field of view and a smaller focal length, which causes significant barrel distortion in the video image, significantly affecting the accuracy of the video ranging calculation.

[0006] Second, video ranging requires the camera's installation angle. However, manual measurement of this angle can lead to errors in actual use, significantly impacting the video measurement data. Furthermore, due to the inconvenience of ship operations, it's difficult to measure the camera's installation angle on board. Furthermore, during ship travel, wind and waves, as well as the ship's turbulence, can cause the camera's installation position to shift, resulting in changes in the camera's installation angle. Summary of the Invention

[0007] The present invention addresses the problems of existing video ranging algorithms, such as large barrel distortion in the video image and camera installation position displacement that easily causes changes in the camera installation angle, resulting in video ranging accuracy that does not meet expectations. The present invention provides an improved ship target ranging method. By automatically correcting camera barrel distortion and automatically calculating the camera installation angle, even videos with barrel distortion can be used for ship ranging, achieving ease of use and high accuracy in ship target ranging. The present invention also relates to an improved ship target ranging system.

[0008] The technical solutions of the present invention are as follows:

[0009] An improved ship target ranging method is characterized by comprising the following steps:

[0010] The automatic correction step of the barrel distortion of the camera is as follows: the three radial distortion parameters, two tangential distortion parameters and four camera intrinsic parameters of the camera are calculated by camera calibration technology. The three radial distortion parameters are first-order, second-order and third-order radial distortion parameters, the two tangential distortion parameters are tangential distortion parameters in the x and y directions of the image coordinate system, and the four camera intrinsic parameters include the focal length of the camera in the x and y directions of the image coordinate system and the coordinates of the image principal point in the x and y directions of the image coordinate system; then, based on the calculated camera intrinsic parameters, radial distortion parameters and tangential distortion parameters, an automatic distortion correction algorithm is used to establish a mapping relationship between the barrel distortion image and the ideal distortion-free image. For each pixel position in the ideal distortion-free image, its corresponding position in the barrel distortion image is calculated by reverse mapping, and then the pixels of the barrel distortion image are mapped to the ideal distortion-free image according to the mapping relationship, thereby achieving preliminary correction of the barrel distortion image, and repairing and filling the gap pixels between the pixels of the preliminary corrected image to generate a distortion-free corrected image;

[0011] The camera installation angle is automatically calculated using a deep learning model to identify the sea level in the camera video image. An automatic distortion correction algorithm is then used to determine distortion of the identified sea level and, if distorted, correct it to a horizontal line. Based on the camera imaging principle, the system combines the video resolution, the camera's vertical field of view, the camera's focal length, the camera's CMOS image sensor height, the pixel distance from the sea level to the horizontal centerline of the video image, and the corresponding CMOS image sensor distance to construct a proportional relationship between the tangent function of the angle between the camera lens direction and the sea level. The inverse tangent function is then used to automatically calculate the angle between the camera lens direction and the camera pole, thereby obtaining the camera installation angle.

[0012] In the ranging optimization step, based on the automatically calculated camera installation angle and the pixel coordinates of the distortion-free corrected image, the video ranging algorithm is used to calculate and update the distance from the camera to the lower edge of the video in the world coordinate system, as well as the longitudinal and lateral distances from the other ship position to the camera. This then obtains the world coordinate system distance from the other ship position on the corrected line of sight video screen to the camera, thereby improving the ship target ranging.

[0013] Preferably, in the automatic correction step of the barrel distortion of the camera, three radial distortion parameters, two tangential distortion parameters and four camera intrinsic parameters of the camera are obtained through camera calibration technology, and the specific steps are as follows:

[0014] S1. Calibrate the camera using a checkerboard calibration plate having a plurality of identical checkerboard cells arranged in multiple rows and columns; measure the width of the checkerboard cells in the checkerboard calibration plate, and calculate the coordinates of the intersection of each checkerboard cell in the world coordinate system;

[0015] S2. Adjust the shooting direction of the camera to obtain chessboard images in different directions and angles, use the findChessBoardCorners function in OpenCV to identify the intersection points of each chessboard unit in the chessboard image, and obtain the coordinates of the intersection points of each chessboard unit in the image coordinate system;

[0016] S3. Input the coordinates of the intersection points of each checkerboard unit in the world coordinate system obtained in step S1, the coordinates of the intersection points of each checkerboard unit in the image coordinate system obtained in step S2, and the resolution of the checkerboard image into the calibrateCamera function in opencv to solve the camera's three radial distortion parameters, two tangential distortion parameters, and four camera intrinsic parameters.

[0017] Preferably, in the automatic correction step of the barrel distortion of the camera, an automatic barrel distortion correction algorithm based on deep learning or a preset mathematical model is used to establish a mapping relationship between the barrel-distorted image and the ideal undistorted image. The automatic barrel distortion correction algorithm maps the pixel positions in the ideal undistorted image to the edge-stretched barrel-distorted image through reverse mapping, and then remaps the pixels of the barrel-distorted image to the ideal positions, restoring the straight line characteristics of the image edge, thereby achieving preliminary correction of the barrel-distorted image.

[0018] After the initial correction of the barrel-distorted image, if there are holes between pixels in the image due to pixel remapping, the mean substitution method or the neighborhood substitution method is used to automatically repair and fill the holes to generate a distortion-free corrected image.

[0019] Preferably, in the step of automatically correcting the barrel distortion of the camera, the automatic distortion correction algorithm first compresses the original video image before the barrel distortion correction so that the resolution of the compressed original video image is consistent with the resolution of the barrel distortion image, and then establishes a mapping relationship between the barrel distortion image and the ideal undistorted image;

[0020] When establishing a mapping relationship between a barrel-distorted image and an ideal undistorted image, if some pixels in the ideal undistorted image cannot find corresponding values ​​in the barrel-distorted image due to reverse mapping, thereby forming leaky pixels, then adjacent pixels of the leaky pixels are determined in the ideal undistorted image. If the adjacent pixels have a corresponding mapping relationship in the barrel-distorted image, the mapping result of the adjacent pixels is used to replace the mapping result of the leaky pixels.

[0021] Preferably, in the step of automatically calculating the camera installation angle, a deep learning model based on the U-Net architecture is used to identify the sea horizon in the camera video image, and the deep learning model is trained by ship video frames containing different sea conditions and lighting conditions.

[0022] Preferably, in the step of automatically calculating the camera installation angle, a proportional relationship formula 1 is constructed based on the camera imaging principle:

[0023] Formula 2 is constructed based on the principle related to the vertical field of view angle:

[0024] According to the definition of the tangent function of angle β, formula 3 is obtained:

[0025] In formula 1-3, HI is the video resolution height, 2γ is the vertical field of view of the camera, F is the focal length of the camera, Ch is the height of the camera's CMOS image sensor, Dp is the pixel distance from the sea level to the horizontal centerline of the image, CY is the distance of the CMOS image sensor corresponding to Dp, and β is the angle between the camera lens direction and the sea level.

[0026] Formula 4 is derived from Formula 1 and Formula 2:

[0027] Formula 5 is derived from Formula 3 and Formula 4:

[0028] Finally, according to Formula 6 is derived:

[0029] In Formula 6, α is the camera installation angle. Thus, the automatically calculated camera installation angle is obtained.

[0030] Preferably, in the ranging optimization step, based on the automatically calculated camera installation angle and the pixel coordinates of the undistorted corrected image, combined with the vertical field of view angle and the camera height of the camera, the distance from the camera to the lower edge of the video in the world coordinate system is calculated and updated by a video ranging algorithm. The distance from the camera to the lower edge of the video is the vertical distance between the optical center of the camera and the vertical projection point of the sea level at the lower edge of the image. Then, based on the distance from the camera to the lower edge of the video, the longitudinal distance and the lateral distance from the position of the other ship to the camera in the world coordinate system are updated, thereby obtaining the world coordinate system distance from the position of the other ship to the camera on the corrected line of sight video screen.

[0031] An improved ship target ranging system is characterized by comprising a camera barrel distortion automatic correction module, a camera installation angle automatic calculation module and a ranging optimization module connected in sequence.

[0032] The camera barrel distortion automatic correction module calculates three radial distortion parameters, two tangential distortion parameters and four camera intrinsic parameters of the camera through camera calibration technology. The three radial distortion parameters are first-order, second-order and third-order radial distortion parameters, respectively. The two tangential distortion parameters are tangential distortion parameters in the x and y directions of the image coordinate system. The four camera intrinsic parameters include the focal length of the camera in the x and y directions of the image coordinate system and the coordinates of the image principal point in the x and y directions of the image coordinate system. Then, based on the calculated camera intrinsic parameters, radial distortion parameters and tangential distortion parameters, an automatic distortion correction algorithm is used to establish a mapping relationship between the barrel distortion image and the ideal undistorted image. For each pixel position in the ideal undistorted image, its corresponding position in the barrel distortion image is calculated through reverse mapping. Then, the pixels of the barrel distortion image are mapped to the ideal undistorted image according to the mapping relationship, thereby achieving preliminary correction of the barrel distortion image. The leaky pixels between the pixels of the preliminary corrected image are repaired and filled to generate a corrected image without distortion.

[0033] The camera installation angle automatic calculation module uses a deep learning model to identify the sea level in the camera video image, and adopts an automatic distortion correction algorithm to determine the distortion of the identified sea level and correct the distortion of the identified sea level into a horizontal straight line when the identified sea level is distorted. According to the camera imaging principle, combined with the video image resolution, the vertical field angle of the camera, the camera focal length, the height of the camera's CMOS image sensor, the pixel distance from the sea level to the horizontal center line of the video image and the corresponding CMOS image sensor distance, a proportional relationship involving the tangent function of the angle between the camera lens direction and the sea level is constructed, and then the inverse tangent function is used to automatically calculate the angle between the camera lens direction and the camera pole, thereby obtaining the camera installation angle.

[0034] The ranging optimization module calculates and updates the distance from the camera to the lower edge of the video in the world coordinate system, as well as the longitudinal and lateral distances from the other ship's position to the camera, based on the automatically calculated camera installation angle and the pixel coordinates of the distortion-free corrected image using a video ranging algorithm. This module then obtains the world coordinate system distance from the other ship's position on the corrected line of sight video screen to the camera, thereby improving ship target ranging.

[0035] Preferably, in the automatic correction module for barrel distortion of a camera, three radial distortion parameters, two tangential distortion parameters and four camera intrinsic parameters of the camera are obtained through camera calibration technology, and the specific steps are as follows:

[0036] S1. Calibrate the camera using a checkerboard calibration plate having a plurality of identical checkerboard cells arranged in multiple rows and columns; measure the width of the checkerboard cells in the checkerboard calibration plate, and calculate the coordinates of the intersection of each checkerboard cell in the world coordinate system;

[0037] S2. Adjust the shooting direction of the camera to obtain chessboard images in different directions and angles, use the findChessBoardCorners function in OpenCV to identify the intersection points of each chessboard unit in the chessboard image, and obtain the coordinates of the intersection points of each chessboard unit in the image coordinate system;

[0038] S3. Input the coordinates of the intersection points of each checkerboard unit in the world coordinate system obtained in step S1, the coordinates of the intersection points of each checkerboard unit in the image coordinate system obtained in step S2, and the resolution of the checkerboard image into the calibrateCamera function in opencv to solve the camera's three radial distortion parameters, two tangential distortion parameters, and four camera intrinsic parameters.

[0039] Preferably, in the automatic barrel distortion correction module of the camera, the automatic distortion correction algorithm first compresses the original video image before barrel distortion correction so that the resolution of the compressed original video image is consistent with the resolution of the barrel distortion image, and then establishes a mapping relationship between the barrel distortion image and the ideal undistorted image;

[0040] When establishing a mapping relationship between a barrel-distorted image and an ideal undistorted image, if some pixels in the ideal undistorted image cannot find corresponding values ​​in the barrel-distorted image due to reverse mapping, thereby forming leaky pixels, then adjacent pixels of the leaky pixels are determined in the ideal undistorted image. If the adjacent pixels have a corresponding mapping relationship in the barrel-distorted image, the mapping result of the adjacent pixels is used to replace the mapping result of the leaky pixels.

[0041] The beneficial effects of the present invention are:

[0042] The present invention provides an improved method for ship target ranging, which can also be called an improved method for ship target ranging. Its camera barrel distortion automatic correction step calculates the camera's three radial distortion parameters, two tangential distortion parameters, and four camera internal parameters to establish a mapping relationship between the distorted image and the ideal image, significantly eliminating barrel distortion caused by the camera lens and improving the image's geometric accuracy. Multi-order parameter correction, which simultaneously considers first-order, second-order, and third-order radial distortion and bidirectional tangential distortion, is more accurate than traditional single-order correction. The method also has a reverse mapping strategy that reversely calculates the distorted image position from the ideal image, avoiding pixel overlap or missing issues that may result from forward mapping. Pixel holes generated during the correction process are repaired and filled, thereby repairing image holes, avoiding image missing caused by mapping discontinuities, and ensuring image integrity. The resulting distortion-free image provides a more accurate pixel-level basis for subsequent steps such as horizon recognition and ship positioning, reducing positioning errors introduced by distortion. Its automatic calculation step of the camera installation angle uses deep learning to identify the sea level, and automatically calculates the installation angle in combination with the camera parameters to adapt to the ship's posture changes such as shaking and tilting during navigation. Even if there is an angle deviation in the initial installation of the camera, or the angle changes due to factors such as vibration during use, the algorithm can automatically calculate the actual angle of the camera, eliminate installation errors, and improve system robustness. Based on the camera imaging principle and geometric optics principle, the proportional relationship is constructed, and the inverse tangent function is used to accurately solve the angle to achieve angle calculation optimization. Compared with traditional manual measurement or fixed parameter methods, it is more flexible and accurate; deep learning is used to automatically identify the sea level, and then combined with the camera imaging model to automatically calculate the angle, realizing the fusion of data-driven and model-driven, without the need for human intervention, and adapting to the dynamic environment of the ship. Its ranging optimization step comprehensively considers the camera angle, distortion correction and pixel coordinates, builds a more comprehensive ranging model, realizes multi-parameter fusion ranging, optimizes the video ranging algorithm, completes distance calculation correction, and significantly improves the longitudinal (distance) and lateral (azimuth) ranging accuracy of ship targets. It maps pixel coordinates to the world coordinate system to achieve the measurement of real physical distance, providing reliable data for applications such as ship collision avoidance and navigation. Through world coordinate system mapping, it directly outputs the real physical distance rather than the relative pixel distance, which is more in line with actual application needs. In addition, due to the automatic calculation of the installation angle, the ranging result can continuously adapt to changes in the ship's posture, has dynamic adaptability, and can maintain high precision.

[0043] The method provided by the present invention deeply integrates computer vision (distortion correction, deep learning) with optical imaging principles. Through the camera barrel distortion automatic correction step (which can be understood as the camera barrel distortion automatic correction algorithm) and the camera installation angle automatic calculation step (which can be understood as the camera installation angle automatic calculation algorithm), it achieves dual optimization of the distortion automatic correction algorithm and the angle automatic calculation algorithm, forming a complete ship target ranging solution, greatly improving the ship target ranging accuracy. Even in complex sea conditions, it can maintain stable performance. It can also automatically adapt to dynamic factors such as camera installation errors and ship shaking, reduce manual intervention, and enhance reliability. This method can improve the ranging accuracy of large-field-of-view, small-focal-length cameras, so that videos after barrel distortion can also be used for ship ranging; at the same time, it reduces the parameters required for video ranging, thereby reducing manual workload and reducing manual measurement errors; its high-precision video ranging algorithm can also provide data and algorithm support for other ship algorithms such as ship automatic surveillance and autonomous driving, playing a key role in intelligent shipping fields such as ship automatic surveillance, collision avoidance warning, and channel monitoring.

[0044] Furthermore, the camera can be calibrated using a checkerboard calibration plate with several identical checkerboard units arranged in multiple rows and columns, that is, a camera calibration method using a single-plane checkerboard, which is between the traditional calibration method and the self-calibration method. This calibration method overcomes the disadvantage of the traditional calibration method requiring a high-precision calibration object. Compared with self-calibration, it improves the accuracy and is easy to operate. Then, by measuring the width of the checkerboard unit and calculating the coordinates of the intersection in the world coordinate system, and using the opencv function to identify the coordinates of the intersection in the image coordinate system, the relevant coordinates and image resolution are finally input into a specific function to solve the parameters. This method can accurately obtain the camera's three radial distortion parameters, two tangential distortion parameters and four camera internal parameters, providing an accurate data basis for subsequent barrel distortion correction; using a checkerboard Calibration operations using the grid calibration board and mature functions in OpenCV are widely used and proven methods in the field of computer vision. They are highly versatile and reliable. They are not restricted by factors such as camera model and brand, and can adapt to a variety of different types of cameras, ensuring that the required parameters can be obtained relatively stably on different devices, thereby improving the applicability of the entire ship target ranging method. Accurate distortion parameters and internal parameters are the key to achieving precise barrel distortion correction. The precise acquisition of these parameters can more accurately establish the mapping relationship between the barrel distortion image and the ideal undistorted image, thereby improving the accuracy of barrel distortion correction and making the subsequently generated undistorted corrected image closer to the real scene, providing more accurate image information for ship target ranging, thereby improving the accuracy of ranging.

[0045] Furthermore, an automatic barrel distortion correction algorithm based on deep learning or a preset mathematical model can be used. By reverse mapping the pixel positions of the ideal undistorted image to the barrel distorted image, and then remapping the barrel distorted image pixels to the ideal positions, the straight line characteristics of the image edge can be effectively restored, and the initial correction of the barrel distorted image can be achieved. This correction method can accurately adjust the characteristics of the barrel distortion at the pixel level, eliminate the image deformation caused by the physical characteristics of the lens, and make the image more consistent with the actual situation in terms of geometry; the leaky pixel points between the pixels of the image after the initial correction are automatically repaired and filled, and a distortion-free correction image can be generated. This operation avoids the problem of image quality degradation caused by missing or discontinuous pixels, ensures the integrity and continuity of the image, and makes the image reach a high level in both visual effect and data integrity, which is more conducive to the accurate implementation of subsequent image-based ship target recognition, ranging and other operations; the generated distortion-free corrected image provides a high-quality image foundation for the subsequent automatic calculation of camera installation angles and ranging optimization steps. High-quality images can make sea level recognition more accurate, and the ranging calculation based on image pixel coordinates is also more reliable, thereby indirectly improving the overall accuracy and stability of ship target ranging and enhancing the performance of the entire ranging method.

[0046] Furthermore, since barrel distortion correction will stretch the image, leaky pixels will be generated between image pixels, resulting in a larger image resolution. Therefore, the image before distortion needs to be compressed to obtain the original image before distortion with the same resolution as the image after distortion. That is to say, in order to improve the efficiency and consistency of subsequent processing, the original video image before barrel distortion correction is compressed so that the resolution of the original video image before barrel distortion is consistent with the resolution of the barrel distortion image, reducing the computational complexity of the subsequent correction algorithm. Especially in real-time video processing scenarios, a lower resolution can significantly reduce the computational burden and improve the system response speed. The unified resolution ensures the consistency of pixel mapping during the distortion correction process, avoids interpolation errors or coordinate mismatches caused by resolution differences, and thus improves the stability and accuracy of the correction algorithm.

[0047] Furthermore, when establishing a mapping relationship between the barrel-distorted image and the ideal undistorted image, if some pixels in the ideal undistorted image cannot find corresponding values ​​in the barrel-distorted image due to reverse mapping, resulting in missing pixels, the adjacent pixels of the missing pixels are determined in the ideal undistorted image. If the adjacent pixels have a corresponding mapping relationship in the barrel-distorted image, the mapping results of the adjacent pixels are used to replace the mapping results of the missing pixels. This can solve the problem of missing pixels caused by coordinate mismatch during the reverse mapping process, eliminate holes or missing pixels in the corrected image, and avoid image breakage caused by mapping failure. Using the mapping results of adjacent valid pixels for replacement ensures the integrity of the mapping relationship, improves image continuity, and ensures that the repaired pixels still conform to the geometric relationship of the original scene without introducing additional distortion. This enables the distortion correction algorithm to handle complex distorted scenes (such as areas with severely deformed edges), expands the scope of application of the algorithm, enhances the robustness of the algorithm, provides a complete and accurate pixel coordinate mapping relationship for subsequent ship target ranging, ensures ranging accuracy, and avoids ranging errors caused by missing pixels.

[0048] For the holes caused by pixel remapping after preliminary correction, the mean substitution method or the neighborhood substitution method is used to fill them, so as to achieve vulnerability repair optimization, effectively eliminate the hole phenomenon in the image, enhance the image integrity, and ensure the visual continuity of the corrected image. The filling method based on local pixel information (such as mean and neighborhood substitution) can retain the local features of the image (such as edges and textures) to the greatest extent, avoid the blurring of details caused by simple interpolation, and provide a more reliable image basis for subsequent ship target recognition and ranging.

[0049] When the original distorted image needs to be restored, the barrel distortion effect is simulated through inverse transformation processing, allowing the system to flexibly switch between the corrected image and the original image, reflecting the data backtracking capability and meeting the needs of certain special application scenarios (such as comparative analysis and fault diagnosis). This technology allows the system to support the output of both corrected data and original format data at the same time, enhancing compatibility with other ship monitoring systems or historical data and expanding the scope of application of the method.

[0050] Users can selectively enable specific optimizations based on actual needs (such as computing resource limitations and application scenarios) to improve the adaptability of the method. The inverse transform technology forms a closed loop with the correction algorithm, ensuring high-precision ranging capabilities for the corrected image while retaining the possibility of retracing the original distortion state, providing a more comprehensive solution for ship target ranging. Furthermore, targeted optimizations are provided at different stages of distortion correction (compression processing, adjacent pixel replacement, and vulnerability repair and filling), forming a complete technology chain. This not only focuses on forward correction, but also implements distortion simulation through inverse transform technology, achieving two-way processing capabilities, breaking through the limitations of traditional correction algorithms' one-way processing.

[0051] Furthermore, a series of formulas can be constructed based on the camera imaging principle and related geometric optical principles. Through parameters such as the video screen resolution height, the camera's vertical field of view angle, and the focal length, the angle between the camera lens direction and the sea level can be accurately calculated, and then the camera installation angle can be obtained. The series of logical deduction processes fully consider the camera's own physical parameters and the relevant information in the image. Compared with the traditional method of determining the installation angle based on experience or rough measurement, the automatic calculation accuracy is greatly improved, providing a more accurate angle reference for subsequent ship target ranging; this improved camera installation angle automatic calculation step can accurately calculate the camera installation angle in real time based on the sea level information in the camera video screen combined with relevant formulas. During the voyage of a ship, the actual angle of the camera may change due to factors such as waves and hull shaking. This method can dynamically adapt to these changes, ensuring that the camera installation angle can be continuously and accurately obtained even in complex sea conditions, thereby ensuring the stability and reliability of ship target ranging. Accurate camera installation angle is one of the key parameters for ship target ranging. By accurately calculating the camera installation angle, the pixel information in the image can be more accurately linked to the actual world coordinate system, reducing the ranging deviation caused by angle error, thereby significantly improving the accuracy of the entire ship target ranging system and providing more reliable data support for applications such as ship collision avoidance and channel monitoring. The proportional relationship formula is constructed based on the camera imaging principle, which theoretically guarantees the scientificity and rationality of the angle calculation. At the same time, in practical applications, by analyzing image features such as the sea horizon and calculating related formulas, it can effectively solve the problem of camera angle changes during actual ship operation, achieving a good combination of theory and practice, making this method more practical and valuable for promotion.

[0052] The present invention also relates to an improved ship target ranging system, which corresponds to the above-mentioned improved ship target ranging method and can be understood as a system for implementing the improved ship target ranging method. The system includes a camera barrel distortion automatic correction module, a camera installation angle automatic calculation module and a ranging optimization module connected in sequence. Through the mutual association and collaborative optimization of multiple modules, distortion correction provides a basis for angle calculation, and accurate calculation of the installation angle further improves the ranging accuracy, forming a closed-loop optimization; at the same time, multi-order distortion parameters and multi-dimensional camera internal parameters are taken into account to achieve parameter refinement processing, which is closer to the real scene than the simplified model; compared with the traditional static ranging system, the present invention significantly improves the performance of the system in the dynamic environment of the ship through a dynamic calculation mechanism. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 It is a flow chart of the improved ship target ranging method of the present invention.

[0054] Figure 2The present invention provides an improved method for measuring distance between ship targets and a camera, and the present invention provides an optimized flow chart for automatically correcting the barrel distortion of the camera. DETAILED DESCRIPTION

[0055] The present invention will be described below with reference to the accompanying drawings.

[0056] The present invention relates to an improved ship target ranging method, the flow chart of which is as follows: Figure 1 As shown, the following steps are included:

[0057] First, the camera barrel distortion automatic correction step uses camera calibration technology to calculate three radial distortion parameters, two tangential distortion parameters, and four camera intrinsic parameters. The three radial distortion parameters are first-order, second-order, and third-order radial distortion parameters, respectively. The two tangential distortion parameters are tangential distortion parameters in the x and y directions of the image coordinate system. The four camera intrinsic parameters include the focal length of the camera in the x and y directions of the image coordinate system and the coordinates of the image principal point in the x and y directions of the image coordinate system. Then, based on the calculated camera intrinsic parameters, radial distortion parameters, and tangential distortion parameters, an automatic distortion correction algorithm is used to establish a mapping relationship between the barrel-distorted image and the ideal undistorted image. For each pixel position in the ideal undistorted image, its corresponding position in the barrel-distorted image is calculated through reverse mapping. Then, the pixels of the barrel-distorted image are mapped to the ideal undistorted image according to the mapping relationship, achieving preliminary correction of the barrel-distorted image. The gap pixels between the pixels of the preliminary corrected image are repaired and filled to generate a distortion-free corrected image.

[0058] The imaging principle of a camera is similar to the pinhole model, where the relationship between the object and the image is a similar triangle. However, in reality, due to factors such as lens perspective distortion, this relationship does not hold true, resulting in an error between the ideal image and the actual image. This error is called distortion. Distortion is mainly categorized into two types: barrel distortion and pincushion distortion.

[0059] In order to capture a wider picture and provide the driver with a wider field of view, the cameras installed on ships generally use cameras with a large field of view and a small focal length, so the imaging distortion is barrel distortion.

[0060] Barrel distortion is a phenomenon in which the image expands outward. While it doesn't affect image clarity, it can affect the positional accuracy of the image, leading to errors or misinterpretations in image measurement. The wider the lens, the greater the barrel distortion, while achieving a wider field of view. Barrel distortion is also nonlinear: minimal at the center of the image, but increasing with distance from the center.

[0061] There are two main methods for correcting barrel distortion: one is to refer to a known standard object, and the other is to refer to a non-standard template with a straight line drawn on it. In the actual use of video ranging in the intelligent ship industry, known standard objects are not easy to find, so the second method is used here: the straight line in the template should remain a straight line after imaging, but it will appear as a curved line after barrel distortion on the image. The deformation parameters are determined during the process of straight line restoration on this curve.

[0062] 1. Camera calibration

[0063] There are many methods for camera calibration, and any method can be used as long as the purpose of distortion correction can be achieved. The purpose of calibration is to obtain the intrinsic parameters, extrinsic parameters, and distortion parameters of the camera. For a monocular camera, since the position of the camera in the world coordinate system is not important, there is no need to calculate the extrinsic parameters of the camera, so here we only need to solve the intrinsic parameters and radial and tangential distortion parameters of the camera. The single-plane checkerboard camera calibration method used here is between the traditional calibration method and the self-calibration method. This calibration method overcomes the shortcomings of the traditional calibration method that requires high-precision calibration objects; compared with self-calibration, it improves accuracy and is easy to operate. Figure 2 The process shown is:

[0064] (1) Prepare a checkerboard calibration plate. You can use a customized high-precision calibration plate, but since the accuracy requirement here is not very high, a standard high-precision checkerboard is not required. You can also prepare an N-row and M-column checkerboard image (N refers to the number of grid intersections in the horizontal direction of the checkerboard, and M refers to the number of grid intersections in the vertical direction of the checkerboard).

[0065] (2) Adjust the direction of the camera and take photos from different directions and angles. The checkerboard positions in the photos should cover as many angles and positions as possible.

[0066] (3) Measure and record the width L (in millimeters) of the checkerboard of the calibration board, and calculate a set of coordinates vecO (in millimeters) of each intersection in the world coordinate system.

[0067] (4) For each image in step 2: use the findChessBoardCorners function in opencv to identify the corners (intersections) of all chessboard squares and record a set of coordinates vecC (unit pixels) of the corners in the image coordinate system.

[0068] (5) Input the coordinates of the intersection point in the world coordinate system vecO obtained in step 3, the coordinates of the intersection point in the image coordinate system vecC obtained in step 4, and the resolution of the calibration image into the calibrateCamera function in opencv, and solve to obtain three radial distortion parameters k1, k2, k3 (first-order, second-order and third-order radial distortion parameters), two axial distortion parameters p1, p2 (tangential distortion parameters in the x and y directions of the image coordinate system), and four camera intrinsic parameters fx, fy (focal length in the x and y directions of the image coordinate system), cx, cy (coordinates of the image principal point in the x and y directions of the image coordinate system).

[0069] Radial distortion parameters (k1, k2, k3): Used to describe barrel distortion or pincushion distortion. Due to limitations in the camera lens manufacturing process, light is refracted differently when passing through different areas of the lens, resulting in radial distortion of the image. k1 is a first-order radial distortion parameter that primarily corrects distortion at the image edges; k2 is a second-order radial distortion parameter used to further correct the degree of distortion; k3, as a higher-order parameter, fine-tunes the accuracy of distortion correction and plays an important role in high-precision scenarios. In ship target ranging, barrel distortion is more severe in cameras with large fields of view and small focal lengths. Accurately acquiring these parameters can effectively correct the image and improve ranging accuracy.

[0070] Axial distortion parameters (p1, p2), also known as tangential distortion parameters, are primarily used to correct for distortion caused by lens installation not being strictly parallel to the imaging plane. Slight deviations in lens installation can cause the image to be misaligned in the horizontal and vertical directions. p1 primarily affects tangential distortion in the x-direction, while p2 primarily affects tangential distortion in the y-direction. During ship travel, the camera may experience slight displacement and angular changes due to wind, waves, and turbulence. Axial distortion parameters help correct for this tangential distortion, ensuring image accuracy.

[0071] Camera intrinsic parameters (fx, fy, cx, cy) are important parameters that describe the internal characteristics of a camera. fx and fy are the camera's focal lengths in the x and y directions, respectively, reflecting the camera's ability to zoom in and out of the image horizontally and vertically. Their values ​​are related to the camera's physical focal length and the pixel size of the image sensor. cx and cy are the coordinates of the image's principal point in the image coordinate system. The principal point is the intersection of the camera's optical axis and the image plane, typically located near the center of the image. These parameters play a key role in converting image pixel coordinates to world coordinates and are the basis for accurate ranging.

[0072] 2. Calculation of image correspondence before and after distortion

[0073] After obtaining all camera parameters (including radial distortion parameters, tangential distortion parameters, and camera intrinsic parameters), we can begin to calculate the correspondence between pixel coordinates before and after distortion. If the image coordinates before distortion (i.e., the pixel coordinates of the ideal undistorted image) are pt1(u, v), the image coordinates after distortion are pt_1(u_d, v_d).

[0074] (1) First, transform the pixel coordinates u and v of the ideal undistorted image to the normalized plane. x and y are the normalized image coordinates of u and v, and x_d and y_d are the normalized image coordinates of u_d and v_d.

[0075] x=(u-cx) / fx

[0076] y=(v-cy) / fy

[0077] r 2 =x 2 +y 2

[0078] x_d=x(1+k1·r 2 +k2·(r 2 ) 2 +k3·(r 2 ) 3 )+2·p2·x·y+p2(r 2 +2x 2 )

[0079] y_d=y(1+k1·r 2 +k2·(r 2 ) 2 +k3·(r 2 ) 3 )+p1·(r 2 +2·y 2 )+2·p2·x·y

[0080] This involves converting the pixel coordinates (u, v) of an ideal, undistorted image into normalized coordinates (x, y). This is a forward mapping operation, the core purpose of which is to remove the influence of internal parameters such as the camera focal length and principal point, allowing subsequent distortion correction and 3D reconstruction operations to be performed at a uniform scale. The normalized coordinates (x, y) are then calculated using the radial distortion parameters k1, k2, k3 and the tangential distortion parameters p1 and p2 to obtain the distorted normalized coordinates (x_d, y_d). The radial distortion parameters compensate for radial deformation caused by distance from the image center, while the tangential distortion parameters compensate for tangential deformation caused by factors such as camera manufacturing. These formulas simulate the effects of barrel distortion on image coordinates. Converting ideal, undistorted normalized coordinates to distorted normalized coordinates is a key computational step in barrel distortion correction, providing the distorted coordinate data for subsequent restoration to the pixel coordinate system and establishing the mapping relationship.

[0081] (2) The coordinates x_d and y_d on the distorted normalized plane are restored to the original pixel coordinate system to obtain the distorted image pixel coordinates u_d and v_d.

[0082] u_d=fx·x_d+cx

[0083] v_d=fy·y_d+cy

[0084] Here, the camera intrinsic parameters fx, fy, cx, and cy are used again to convert the normalized coordinates back to the pixel positions in the actual image, completing the conversion from the normalized coordinate system to the pixel coordinate system. This allows the coordinates calculated after distortion to correspond to the pixels in the actual image. This is an important step in establishing a complete mapping relationship between pixel coordinates before and after distortion, laying the foundation for subsequent generation of a correspondence table and meeting application requirements such as ranging algorithms.

[0085] (3) According to the formula, the coordinates u_d and v_d of the distorted pixel points corresponding to each pre-distorted pixel point u and v have been calculated, thus obtaining the pixel correspondence table map_ud and map_vd of the entire image. However, the distance measurement algorithm requires: given the coordinates u_d and v_d of the distorted pixel points, the pre-distorted pixel points u and v must be calculated. Therefore, another conversion calculation is performed here to obtain the correspondence table map_u and map_v. In other words, the pixels of the barrel distorted image are remapped to the ideal position. By querying this table, the pre-distorted coordinates u and v corresponding to any point coordinates u_d and u_d can be obtained.

[0086] That is, step (3) involves the establishment of a mapping relationship and reverse mapping, specifically including:

[0087] Generation of the forward mapping table: Through steps (1) and (2), the forward mapping relationship from the pixel coordinates u and v of the ideal undistorted image to the pixel coordinates u_d and v_d of the distorted image has been calculated. In step (3), these correspondences are stored as mapping tables map_ud and map_vd, i.e., map_ud[u,v] = u_d and map_vd[u,v] = v_d. This step explicitly establishes the mapping relationship between the ideal undistorted image and the distorted image.

[0088] Construction of the reverse mapping table: Since practical applications require the search for the corresponding ideal undistorted image pixel coordinates u and v based on the distorted image pixel coordinates u_d and v_d, the reverse mapping tables map_u and map_v are further generated, so that map_u[u_d,v_d]=u and map_v[u_d,v_d]=v. This process realizes the reverse mapping from the distorted image to the ideal undistorted image, that is, remapping the pixels of the barrel distorted image to the pixel positions in the ideal undistorted image.

[0089] Restoring straight-line characteristics: Through the reverse mapping tables map_u and map_v, each pixel coordinate u_d and v_d in the distorted image can be found at its corresponding position u and v in the ideal undistorted image. Originally curved edges (such as straight lines) in the distorted image are remapped back to their ideal positions, restoring the straight-line characteristics of the image edges and achieving preliminary correction.

[0090] (4) A problem arises here: the resolution of the image composed of all u_d and v_d point coordinates before distortion increases. This is because barrel distortion correction stretches the image, creating gaps between pixels, which increases the image resolution. Therefore, it is necessary to compress the image before distortion correction (the original video image) to obtain the original image before distortion with the same resolution as the distorted image.

[0091] (5) Another problem is that when establishing the mapping relationship between the barrel-distorted image and the ideal undistorted image, some pixels in the ideal undistorted image cannot find corresponding values ​​in the barrel-distorted image, forming leaky pixels. In other words, the leaky pixels between the pixels of the corrected image will cause some u_d and v_d to not find corresponding u and v. Then we can consider finding a point (adjacent pixel) u_d1 and v_d1 that is closest to u_d and v_d, and use the mapping result of this adjacent pixel to replace the mapping result of the leaky pixel.

[0092] (6) If some algorithms require more than just u and v, and also need to restore the video image before distortion, then a filling algorithm can be designed to fill the gaps between the corrected image pixels, thereby obtaining the original image before correction. The filling method can use the mean replacement method or the neighborhood replacement method: scan each pixel point, take its 8-neighborhood or 16-neighborhood of equal size, and perform mean filling or select one of the points with a value for replacement filling.

[0093] 2. Automatic calculation steps for the camera installation angle: Use a deep learning model to identify the sea horizon in the camera video screen, and adopt an automatic distortion correction algorithm to judge the distortion of the identified sea horizon and correct it into a horizontal straight line when the identified sea horizon is distorted; according to the camera imaging principle, combined with the video screen resolution, the camera's vertical field of view angle, the camera focal length, the camera's CMOS image sensor height, the pixel distance from the sea horizon to the horizontal center line of the video screen and its corresponding CMOS image sensor distance, construct a proportional relationship involving the tangent function of the angle between the camera lens direction and the sea level, and then use the inverse tangent function to dynamically calculate the angle between the camera lens direction and the camera pole to obtain the camera installation angle.

[0094] A commonly used method for calculating camera angles is to infer the camera's installation angle based on the actual length or width of a known object within the image, as well as the pixel size of the object within the image. However, suitable reference objects of known length are difficult to find, and manual on-site measurements are required, increasing the workload. The inconvenience of working onboard also impacts the measurement process. Furthermore, the closer the reference object, the greater the error in camera angle calculation caused by a small measurement error, which has a greater impact on the video measurement data. Therefore, the present invention has designed a camera angle measurement method using the sea horizon as a reference object.

[0095] (1) Using a deep learning model, the sea-sky boundary, i.e., the dividing line between the sea and the sky (the sea horizon), is identified. Due to camera distortion, this dividing line is generally arc-shaped, so the distortion correction algorithm from the previous step is used to correct the identified sea horizon into a horizontal straight line. Furthermore, a deep learning model based on a U-Net architecture can be used to identify the sea horizon in the camera video. The deep learning model is trained on ship video frames containing different sea conditions and lighting conditions.

[0096] (2) Based on the following data: video resolution WI*HI, camera vertical field of view angle 2γ, camera focal length F, Dp is the pixel distance from the sea level to the horizontal center line of the image, Ch is the height of the camera's CMOS image sensor, CY is the distance of the CMOS image sensor corresponding to Dp, calculate the angle α between the camera lens direction and the camera pole (then the angle between the camera lens direction and the sea level is ). Among them, WI, HI, 2γ, F, and Dp are all known data, while Ch, CY, β, and α are all unknown data.

[0097] Based on the camera imaging principle, the proportional relationship is constructed and formula 1 is obtained:

[0098] Formula 2 is constructed based on the principle related to the vertical field of view angle:

[0099] According to the definition of the tangent function of angle β, formula 3 is obtained:

[0100] Formula 1 and Formula 2 lead to Formula 4:

[0101] Formula 3 and Formula 4 derive Formula 5:

[0102] Finally, we get formula 6:

[0103] At this point, the camera installation angle α can be calculated.

[0104] 3. Distance measurement optimization step: Based on the automatically calculated camera installation angle and the pixel coordinates of the distortion-free corrected image, the video ranging algorithm is used to calculate and update the distance from the camera to the lower edge of the video in the world coordinate system, as well as the longitudinal and lateral distances from the other ship's position to the camera. This then obtains the world coordinate system distance from the other ship's position on the corrected line of sight video screen to the camera, thereby improving the ship target ranging.

[0105] Based on the above two optimizations, we first obtained a more accurate camera installation angle α, and at the same time determined the corresponding point coordinates pt1(u, v) of the point pt_1(u_d, v_d) in the distorted image in the undistorted corrected image.

[0106] (1) The camera installation angle α is a very important value in the monocular ranging algorithm. The upgrade of the camera installation angle automatic calculation algorithm reduces the workload of manual on-site measurement and improves the accuracy of the camera angle calculation value, which is equivalent to improving the accuracy of monocular ranging.

[0107] (2) For the point pt1_1 in the video screen, find its corresponding point coordinate pt1 before distortion. In the prior art, assuming that there is a target coordinate pt_1 on the video screen, the world coordinate system distance Do from the point pt_1 to the camera is calculated. Then, after the upgrade of the method of the present invention, the point pt_1 must first find the corresponding point coordinate pt1 before distortion, that is, update pt_1 to pt1, and calculate the world coordinate system distance Do1 from the point pt1 to the camera. That is, after the algorithm upgrade of the present invention, the world coordinate system distance Do from the point pt_1 on the video screen to the camera calculated by the original video ranging algorithm of the prior art is corrected to Do1, thereby improving the accuracy of monocular ranging.

[0108] Specifically, based on the automatically calculated camera installation angle and the pixel coordinates of the undistorted corrected image, combined with the camera's vertical field of view and camera height, the distance from the camera to the lower edge of the video in the world coordinate system is calculated and updated using the existing video ranging algorithm. The distance from the camera to the lower edge of the video is the vertical distance between the optical center of the camera and the vertical projection point of the sea level at the lower edge of the image. Then, based on the distance from the camera to the lower edge of the video, the longitudinal and lateral distances from the other ship's position to the camera in the world coordinate system are updated, thereby obtaining the world coordinate system distance Do1 from the other ship's position to the camera on the corrected line of sight video screen.

[0109] The present invention achieves dual optimization of the improvement of the automatic distortion correction algorithm and the automatic angle calculation algorithm through the automatic correction step of the camera barrel distortion (i.e., the automatic correction algorithm of the camera barrel distortion) and the automatic calculation step of the camera installation angle (i.e., the automatic calculation algorithm of the camera installation angle), thereby forming a complete ship target ranging solution and providing data and algorithm support for other ships. It can also be integrated into the existing automatic observation algorithm, update the relevant algorithm steps, improve the target ranging accuracy, and in the field of ships, monitor ships on the sea in real time and display ship-related information, assist drivers in observing sea conditions around the clock, help crew members to be more vigilant, achieve 24-hour automatic observation, and improve the accuracy of automatic observation.

[0110] The present invention also relates to an improved ship target ranging system, which corresponds to the above-mentioned improved ship target ranging method and can be understood as a system for implementing the improved ship target ranging method, comprising a camera barrel distortion automatic correction module, a camera installation angle automatic calculation module and a ranging optimization module connected in sequence, wherein the camera barrel distortion automatic correction module calculates three radial distortion parameters, two tangential distortion parameters and four camera internal parameters of the camera through camera calibration technology, wherein the three radial distortion parameters are first-order, second-order and third-order radial distortion parameters, respectively, and the two tangential distortion parameters are tangential distortion parameters in the x and y directions of the image coordinate system. The four camera intrinsic parameters include the focal length of the camera in the x and y directions of the image coordinate system and the coordinates of the image principal point in the x and y directions of the image coordinate system; then, based on the calculated camera intrinsic parameters, radial distortion parameters, and tangential distortion parameters, an automatic distortion correction algorithm is used to establish a mapping relationship between the barrel-distorted image and the ideal undistorted image. For each pixel position in the ideal undistorted image, its corresponding position in the barrel-distorted image is calculated by reverse mapping, and then the pixels of the barrel-distorted image are mapped to the ideal undistorted image according to the mapping relationship, thereby achieving preliminary correction of the barrel-distorted image, and repairing and filling the missing pixels between the pixels of the image after the preliminary correction to generate a corrected image without distortion;

[0111] The camera installation angle automatic calculation module uses a deep learning model to identify the sea level in the camera video image, and adopts an automatic distortion correction algorithm to determine the distortion of the identified sea level and correct the distortion of the identified sea level into a horizontal straight line when the identified sea level is distorted. According to the camera imaging principle, combined with the video image resolution, the vertical field angle of the camera, the camera focal length, the height of the camera's CMOS image sensor, the pixel distance from the sea level to the horizontal center line of the video image and the corresponding CMOS image sensor distance, a proportional relationship involving the tangent function of the angle between the camera lens direction and the sea level is constructed, and then the inverse tangent function is used to automatically calculate the angle between the camera lens direction and the camera pole, thereby obtaining the camera installation angle.

[0112] The ranging optimization module calculates and updates the distance from the camera to the lower edge of the video in the world coordinate system, as well as the longitudinal and lateral distances from the other ship's position to the camera, based on the automatically calculated camera installation angle and the pixel coordinates of the distortion-free corrected image using a video ranging algorithm. This module then obtains the world coordinate system distance from the other ship's position on the corrected line of sight video screen to the camera, thereby improving ship target ranging.

[0113] Furthermore, in the automatic correction module for barrel distortion of the camera, three radial distortion parameters, two tangential distortion parameters and four camera intrinsic parameters of the camera are obtained through camera calibration technology. The specific steps are as follows:

[0114] S1. Calibrate the camera using a checkerboard calibration plate having a plurality of identical checkerboard cells arranged in multiple rows and columns; measure the width of the checkerboard cells in the checkerboard calibration plate, and calculate the coordinates of the intersection of each checkerboard cell in the world coordinate system;

[0115] S2. Adjust the shooting direction of the camera to obtain chessboard images in different directions and angles, use the findChessBoardCorners function in OpenCV to identify the intersection points of each chessboard unit in the chessboard image, and obtain the coordinates of the intersection points of each chessboard unit in the image coordinate system;

[0116] S3. Input the coordinates of the intersection points of each checkerboard unit in the world coordinate system obtained in step S1, the coordinates of the intersection points of each checkerboard unit in the image coordinate system obtained in step S2, and the resolution of the checkerboard image into the calibrateCamera function in opencv to solve the camera's three radial distortion parameters, two tangential distortion parameters, and four camera intrinsic parameters.

[0117] Furthermore, in the camera barrel distortion automatic correction module, the distortion automatic correction algorithm first compresses the original video image before barrel distortion correction so that the resolution of the compressed original video image is consistent with the resolution of the barrel distortion image, and then establishes a mapping relationship between the barrel distortion image and the ideal undistorted image;

[0118] When establishing a mapping relationship between a barrel-distorted image and an ideal undistorted image, if some pixels in the ideal undistorted image cannot find corresponding values ​​in the barrel-distorted image due to reverse mapping, thereby forming leaky pixels, then adjacent pixels of the leaky pixels are determined in the ideal undistorted image. If the adjacent pixels have a corresponding mapping relationship in the barrel-distorted image, the mapping result of the adjacent pixels is used to replace the mapping result of the leaky pixels.

[0119] It should be noted that the specific embodiments described above can enable those skilled in the art to more fully understand the present invention, but do not limit the present invention in any way. Therefore, although this specification has described the present invention in detail with reference to the drawings and embodiments, those skilled in the art should understand that the present invention can still be modified or replaced with equivalents. In short, all technical solutions and improvements that do not depart from the spirit and scope of the present invention should be included in the scope of protection of the patent for the present invention.

Claims

1. An improved ship target ranging method, characterized in that: The following steps are involved: The automatic correction step of the barrel distortion of the camera is as follows: the three radial distortion parameters, two tangential distortion parameters and four camera intrinsic parameters of the camera are calculated by camera calibration technology. The three radial distortion parameters are first-order, second-order and third-order radial distortion parameters, the two tangential distortion parameters are tangential distortion parameters in the x and y directions of the image coordinate system, and the four camera intrinsic parameters include the focal length of the camera in the x and y directions of the image coordinate system and the coordinates of the image principal point in the x and y directions of the image coordinate system; then, based on the calculated camera intrinsic parameters, radial distortion parameters and tangential distortion parameters, an automatic distortion correction algorithm is used to establish a mapping relationship between the barrel distortion image and the ideal distortion-free image. For each pixel position in the ideal distortion-free image, its corresponding position in the barrel distortion image is calculated by reverse mapping, and then the pixels of the barrel distortion image are mapped to the ideal distortion-free image according to the mapping relationship, thereby achieving preliminary correction of the barrel distortion image, and repairing and filling the gap pixels between the pixels of the preliminary corrected image to generate a distortion-free corrected image; The camera installation angle is automatically calculated using a deep learning model to identify the sea level in the camera video image. An automatic distortion correction algorithm is then used to determine distortion of the identified sea level and, if distorted, correct it to a horizontal line. Based on the camera imaging principle, the system combines the video resolution, the camera's vertical field of view, the camera's focal length, the camera's CMOS image sensor height, the pixel distance from the sea level to the horizontal centerline of the video image, and the corresponding CMOS image sensor distance to construct a proportional relationship between the tangent function of the angle between the camera lens direction and the sea level. The inverse tangent function is then used to automatically calculate the angle between the camera lens direction and the camera pole, thereby obtaining the camera installation angle. In the ranging optimization step, based on the automatically calculated camera installation angle and the pixel coordinates of the distortion-free corrected image, the video ranging algorithm is used to calculate and update the distance from the camera to the lower edge of the video in the world coordinate system, as well as the longitudinal and lateral distances from the other ship position to the camera. This then obtains the world coordinate system distance from the other ship position on the corrected line of sight video screen to the camera, thereby improving the ship target ranging.

2. The improved ship target ranging method according to claim 1, characterized in that: In the automatic correction step of the barrel distortion of the camera, three radial distortion parameters, two tangential distortion parameters and four camera intrinsic parameters of the camera are obtained through camera calibration technology. The specific steps are as follows: S1. Calibrate the camera using a checkerboard calibration plate having a plurality of identical checkerboard cells arranged in multiple rows and columns; measure the width of the checkerboard cells in the checkerboard calibration plate, and calculate the coordinates of the intersection of each checkerboard cell in the world coordinate system; S2. Adjust the shooting direction of the camera to obtain chessboard images in different directions and angles, use the findChessBoardCorners function in OpenCV to identify the intersection points of each chessboard unit in the chessboard image, and obtain the coordinates of the intersection points of each chessboard unit in the image coordinate system; S3. Input the coordinates of the intersection points of each checkerboard unit in the world coordinate system obtained in step S1, the coordinates of the intersection points of each checkerboard unit in the image coordinate system obtained in step S2, and the resolution of the checkerboard image into the calibrateCamera function in opencv to solve the camera's three radial distortion parameters, two tangential distortion parameters, and four camera intrinsic parameters.

3. The improved ship target ranging method according to claim 1, characterized in that: In the automatic correction of the barrel distortion of the camera, an automatic barrel distortion correction algorithm based on deep learning or a preset mathematical model is used to establish a mapping relationship between the barrel-distorted image and the ideal undistorted image. The automatic barrel distortion correction algorithm maps the pixel positions in the ideal undistorted image to the edge-stretched barrel-distorted image through reverse mapping, and then remaps the pixels of the barrel-distorted image to the ideal positions, restoring the straight line characteristics of the image edge, thereby achieving preliminary correction of the barrel-distorted image. After the initial correction of the barrel-distorted image, if there are holes between pixels in the image due to pixel remapping, the mean substitution method or the neighborhood substitution method is used to automatically repair and fill the holes to generate a distortion-free corrected image.

4. The improved ship target ranging method according to claim 1, characterized in that: In the automatic correction step of the barrel distortion of the camera, the automatic distortion correction algorithm first compresses the original video image before the barrel distortion correction so that the resolution of the compressed original video image is consistent with the resolution of the barrel distortion image, and then establishes a mapping relationship between the barrel distortion image and the ideal undistorted image; When establishing a mapping relationship between a barrel-distorted image and an ideal undistorted image, if some pixels in the ideal undistorted image cannot find corresponding values ​​in the barrel-distorted image due to reverse mapping, thereby forming leaky pixels, then adjacent pixels of the leaky pixels are determined in the ideal undistorted image. If the adjacent pixels have a corresponding mapping relationship in the barrel-distorted image, the mapping result of the adjacent pixels is used to replace the mapping result of the leaky pixels.

5. The improved ship target ranging method according to any one of claims 1 to 4, characterized in that: In the step of automatically calculating the camera installation angle, a deep learning model based on the U-Net architecture is used to identify the sea horizon in the camera video image. The deep learning model is trained with ship video frames containing different sea conditions and lighting conditions.

6. The improved ship target ranging method according to claim 5, characterized in that: In the automatic calculation step of the camera installation angle, the proportional relationship formula 1 is constructed based on the camera imaging principle: Formula 2 is constructed based on the principle related to the vertical field of view angle: According to the definition of the tangent function of angle β, formula 3 is obtained: In formula 1-3, HI is the video resolution height, 2γ is the vertical field of view of the camera, F is the focal length of the camera, Ch is the height of the camera's CMOS image sensor, Dp is the pixel distance from the sea level to the horizontal centerline of the image, CY is the distance of the CMOS image sensor corresponding to Dp, and β is the angle between the camera lens direction and the sea level. Formula 4 is derived from Formula 1 and Formula 2: Formula 5 is derived from Formula 3 and Formula 4: Finally, according to Formula 6 is derived: In Formula 6, α is the camera installation angle. Thus, the automatically calculated camera installation angle is obtained.

7. The improved ship target ranging method according to any one of claims 1 to 4, characterized in that: In the ranging optimization step, based on the automatically calculated camera installation angle and the pixel coordinates of the undistorted corrected image, combined with the camera's vertical field of view and camera height, a video ranging algorithm is used to calculate and update the distance from the camera to the lower edge of the video in the world coordinate system. The distance from the camera to the lower edge of the video is the vertical distance between the camera's optical center and the vertical projection point of the sea level at the lower edge of the image. Based on the distance from the camera to the lower edge of the video, the longitudinal and lateral distances from the other ship's position to the camera in the world coordinate system are then updated, thereby obtaining the world coordinate system distance from the other ship's position to the camera on the corrected line of sight video screen.

8. An improved ship target ranging system, characterized in that: It includes a camera barrel distortion automatic correction module, a camera installation angle automatic calculation module and a distance measurement optimization module connected in sequence. The camera barrel distortion automatic correction module calculates three radial distortion parameters, two tangential distortion parameters and four camera intrinsic parameters of the camera through camera calibration technology. The three radial distortion parameters are first-order, second-order and third-order radial distortion parameters, respectively. The two tangential distortion parameters are tangential distortion parameters in the x and y directions of the image coordinate system. The four camera intrinsic parameters include the focal length of the camera in the x and y directions of the image coordinate system and the coordinates of the image principal point in the x and y directions of the image coordinate system. Then, based on the calculated camera intrinsic parameters, radial distortion parameters and tangential distortion parameters, an automatic distortion correction algorithm is used to establish a mapping relationship between the barrel distortion image and the ideal undistorted image. For each pixel position in the ideal undistorted image, its corresponding position in the barrel distortion image is calculated through reverse mapping. Then, the pixels of the barrel distortion image are mapped to the ideal undistorted image according to the mapping relationship, thereby achieving preliminary correction of the barrel distortion image. The leaky pixels between the pixels of the preliminary corrected image are repaired and filled to generate a corrected image without distortion. The camera installation angle automatic calculation module uses a deep learning model to identify the sea level in the camera video image, and adopts an automatic distortion correction algorithm to determine the distortion of the identified sea level and correct the distortion of the identified sea level into a horizontal straight line when the identified sea level is distorted. According to the camera imaging principle, combined with the video image resolution, the vertical field angle of the camera, the camera focal length, the height of the camera's CMOS image sensor, the pixel distance from the sea level to the horizontal center line of the video image and the corresponding CMOS image sensor distance, a proportional relationship involving the tangent function of the angle between the camera lens direction and the sea level is constructed, and then the inverse tangent function is used to automatically calculate the angle between the camera lens direction and the camera pole, thereby obtaining the camera installation angle. The ranging optimization module calculates and updates the distance from the camera to the lower edge of the video in the world coordinate system, as well as the longitudinal and lateral distances from the other ship's position to the camera, based on the automatically calculated camera installation angle and the pixel coordinates of the distortion-free corrected image using a video ranging algorithm. This module then obtains the world coordinate system distance from the other ship's position on the corrected line of sight video screen to the camera, thereby improving ship target ranging.

9. The improved ship target ranging system according to claim 8, characterized in that: In the automatic correction module for camera barrel distortion, three radial distortion parameters, two tangential distortion parameters, and four camera intrinsic parameters of the camera are obtained through camera calibration technology. The specific steps are as follows: S1. Calibrate the camera using a checkerboard calibration plate having a plurality of identical checkerboard cells arranged in multiple rows and columns; measure the width of the checkerboard cells in the checkerboard calibration plate, and calculate the coordinates of the intersection of each checkerboard cell in the world coordinate system; S2. Adjust the shooting direction of the camera to obtain chessboard images in different directions and angles, use the findChessBoardCorners function in OpenCV to identify the intersection points of each chessboard unit in the chessboard image, and obtain the coordinates of the intersection points of each chessboard unit in the image coordinate system; S3. Input the coordinates of the intersection points of each checkerboard unit in the world coordinate system obtained in step S1, the coordinates of the intersection points of each checkerboard unit in the image coordinate system obtained in step S2, and the resolution of the checkerboard image into the calibrateCamera function in opencv to solve the camera's three radial distortion parameters, two tangential distortion parameters, and four camera intrinsic parameters.

10. The improved ship target ranging system according to claim 8 or 9, characterized in that: In the camera barrel distortion automatic correction module, the distortion automatic correction algorithm first compresses the original video image before barrel distortion correction so that the resolution of the compressed original video image is consistent with the resolution of the barrel distortion image, and then establishes a mapping relationship between the barrel distortion image and the ideal undistorted image; When establishing a mapping relationship between a barrel-distorted image and an ideal undistorted image, if some pixels in the ideal undistorted image cannot find corresponding values ​​in the barrel-distorted image due to reverse mapping, thereby forming leaky pixels, then adjacent pixels of the leaky pixels are determined in the ideal undistorted image. If the adjacent pixels have a corresponding mapping relationship in the barrel-distorted image, the mapping result of the adjacent pixels is used to replace the mapping result of the leaky pixels.

Images