Ship freeboard identification method based on zoom camera and laser ranging

Abstract

The application discloses a ship freeboard identification method based on a zoom camera and laser ranging, and comprises the following steps: installing a laser ranging module on a zoom camera holder, rotating the laser ranging module together with the zoom camera holder, and making the distance from the laser ranging module to a ship consistent with the distance from the zoom camera holder to the ship; calculating the edge value of an image containing a ship freeboard in a vertical direction by using a cumulative difference method, finding out two maximum wave crest values from all the edge values, and obtaining a ship water boundary line and a deck line and their respective line indexes in the image containing the ship freeboard according to the two maximum wave crest values; calculating the longitudinal distance between the ship water boundary line and the zoom camera holder according to a zoom camera holder and world coordinate system transformation model; measuring the distance between the laser ranging module and the ship water boundary line by using the laser ranging module, obtaining the longitudinal distance between the ship water boundary line and the zoom camera holder according to the Pythagorean theorem, and then obtaining zoom camera holder picture parameters, so as to obtain the ship freeboard height.

Description

Technical Field

[0001] This invention relates to a ship freeboard identification method based on zoom camera and laser rangefinder, belonging to the field of freeboard identification technology for waterway vessels. Background Technology

[0002] In recent years, with the rapid development of the shipping industry and the increasing demand for nighttime operations, nighttime ship identification and directional lighting technologies have become particularly important. The ship's freeboard is a marker on board used to measure its draft and load, and its accurate identification is crucial for safe navigation, port management, and maritime law enforcement.

[0003] In recent years, technologies for ship detection and identification have become increasingly mature, but technologies related to ship freeboard identification are relatively scarce. CN201811452126.7 proposes an inland waterway freeboard detection method based on a deep reinforcement neural network. This method uses a combination of lidar and a gimbal to scan ships and then uses a reinforcement neural network to detect the ship's freeboard to determine if the ship is overloaded. This method uses lidar as the primary sensor, but due to the inherent limitations of lidar technology, it only covers inland waterways with a channel width of less than 200 meters and cannot detect ships at greater distances. Summary of the Invention

[0004] The technical problem to be solved by this invention is to provide a ship freeboard identification method based on zoom camera and laser ranging, which uses a laser ranging module to replace lidar, breaks through the limit of lidar identification distance of 200 meters, and the maximum identification distance can reach 1000 meters, so as to achieve accurate identification of ship freeboard.

[0005] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0006] The ship freeboard identification method based on zoom camera and laser ranging includes the following steps:

[0007] Step 1: Install a zoom gimbal camera on one side of the channel to capture images of the freeboard of ships in the channel; install a laser ranging module on the zoom gimbal camera so that the laser ranging module rotates with the zoom gimbal camera and the distance from the laser ranging module to the ship is the same as the distance from the zoom gimbal camera to the ship.

[0008] Step 2: Use a zoom gimbal camera to capture an image containing the ship's freeboard. Use the cumulative difference method to calculate the edge values ​​in the vertical direction of the image containing the ship's freeboard. Find the two largest peak values ​​from all edge values. Based on the two largest peak values, obtain the ship's waterline and deck line, as well as their respective row indices in the image containing the ship's freeboard.

[0009] Step 3: Calculate the longitudinal distance between the ship's waterline and the zoom pan-tilt camera according to the transformation model between the zoom pan-tilt camera and the world coordinate system. The calculation formula is as follows:

[0010]

[0011] where Y is the longitudinal distance between the ship's waterline and the zoom pan-tilt camera, H is the installation height of the laser ranging module and the camera, F p is the camera internal parameter, y1 is the row index of the ship's waterline in the image containing the ship's freeboard, and yp is the zoom pan-tilt camera frame parameter;

[0012] Step 4: Measure the distance D between the laser ranging module and the ship's waterline using the laser ranging module. According to the installation height of the laser ranging module and the Pythagorean theorem, obtain the longitudinal distance between the ship's waterline and the zoom pan-tilt camera, that is:

[0013]

[0014] Step 5: According to Step 3 and Step 4, the calculation formula for the zoom pan-tilt camera frame parameter is:

[0015]

[0016] Thus, the height of the ship's freeboard is:

[0017]

[0018] where Y g represents the height of the freeboard of the ship underway, and y2 is the row index of the deck line in the image containing the ship's freeboard.

[0019] As a preferred embodiment of the present invention, the specific process of Step 2 is as follows:

[0020] 1) Set the height of the image containing the ship's freeboard taken by the zoom pan-tilt camera as h and the width as w. Let G(i, j) represent the gray value of the pixel at the i-th row and j-th column of the image, where 0 < G(i, j) < 255. Calculate the mean value of the gray values of each row of the image to obtain the column vector C, that is:

[0021]

[0022] where C(i) represents the mean value of the gray values of all pixels in the i-th row;

[0023] 2) Set the widths of both the ship's waterline and the deck line as b pixels. Perform a difference on C(i) to obtain Diff1(i):

[0024] Diff1(i) = C(i + b) - C(i) i = 1, 2,..., h - b

[0025] Where Diff1(i) represents the value obtained by differentiating C(i), and C(i+b) represents the mean of the gray values ​​of all pixels in the (i+b)th row;

[0026] 3) Calculate the cumulative value of Diff1(i):

[0027]

[0028] Where DiffAcc(i) represents the cumulative value of Diff1(i), Diff1(k) represents the value obtained by differentiating C(k), and C(k) represents the mean of the gray values ​​of all pixels in the k-th row;

[0029] 4) Then perform a difference operation on DiffAcc(i) to obtain Diff2(i):

[0030] Diff2(i)=DiffAcc(i+b)-DiffAcc(i)i=1,2,…,hb

[0031] Where Diff2(i) represents the value obtained by differentiating DiffAcc(i), DiffAcc(i+b) represents the cumulative value of Diff1(i+b), and Diff1(i+b) represents the value obtained by differentiating C(i+b);

[0032] 5) Find the maximum and second largest values ​​from all Diff2(i), and find the row indices corresponding to the maximum and second largest values ​​in the image containing the ship's freeboard. Use the smaller of the two row indices as the index value of the ship's waterline boundary and the larger value as the index value of the deck line.

[0033] As a preferred embodiment of the present invention, the zoom gimbal camera is model iDS-2DY9840IAX-ASP(T5), and the laser ranging module is model LSP-LRS-0510A.

[0034] A computer device includes a memory, a processor, and a computer program stored in the memory and capable of running on the processor, wherein the processor executes the computer program to implement the steps of the ship freeboard identification method based on zoom camera and laser ranging as described above.

[0035] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the ship freeboard identification method based on a zoom camera and laser ranging as described above.

[0036] Compared with the prior art, the present invention, employing the above technical solution, has the following technical effects:

[0037] 1. This invention uses a laser ranging module to replace lidar, breaking through the lidar's 200-meter recognition distance limit and achieving a maximum recognition distance of 1000 meters, thus enabling accurate identification of ship freeboard.

[0038] 2. The method of the present invention can accurately measure the freeboard height of a ship, overcoming the problems of short measurement distance and low accuracy of traditional methods. The freeboard of a ship can be measured up to 1 km away with an accuracy of ±5 cm. Attached Figure Description

[0039] Figure 1 This is a flowchart of the method of the present invention;

[0040] Figure 2 This is a diagram illustrating the implementation effect of the present invention. Detailed Implementation

[0041] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0042] like Figure 1 The diagram shows a flowchart of the ship freeboard identification method based on zoom camera and laser ranging proposed in this invention. The specific steps are as follows:

[0043] Step 1: Mount the laser ranging module onto the zoom gimbal camera, and rotate the laser ranging module together with the zoom gimbal camera to ensure that the distance from the laser to the ship is consistent with the distance from the camera to the ship. The installation height of the camera and the laser ranging module is H.

[0044] Step 2: Use a zoom gimbal camera to capture an image containing the ship's freeboard. Since the ship-water boundary and deck line of the freeboard have significant edge features, the cumulative difference method is used to calculate the edge values ​​in the vertical direction of the image. The peaks of the two edge values ​​are then identified as the ship-water boundary and deck line. The specific process is as follows:

[0045] Assuming the image has height h and width w, and G(i,j) represents the grayscale value of the pixel in row i and column j, where 0 < h < w < g, G(i,j) represents the grayscale value of the pixel in row i and column j. <G(i,j)<255:

[0046] 2.1. Calculate the mean of each row of the image to obtain a column vector C, i.e.

[0047]

[0048] 2.2, Let the width of the ship's waterline boundary and deck line be b pixels. Difference C(i) yields Diff1(i):

[0049] Diff1(i)=C(i+b)-C(i)i=1,2,…,hb

[0050] 2.3 Calculate the cumulative value of Diff1(i):

[0051]

[0052] 2.4, by taking the difference of DiffAcc(i) again, we get:

[0053] Diff2(i)=DiffAcc(i+b)-Diffccc(i)i=1,2,…,hb

[0054] 2.5 Find the two maximum peak values ​​of Diff2(i) and their indices, so as to obtain the row indices y1 and y2 of the horizontal edge.

[0055] Step 3: Based on the positioning results from Step 2, the ship's waterline is y1, the deck line is y2, and the relationship between the ship's waterline and the longitudinal distance Y of the camera can be calculated using the camera-world coordinate system transformation model:

[0056]

[0057] Among them, F p yp is the camera's intrinsic parameter, a fixed value; yp is the image parameter, which changes as the gimbal rotates.

[0058] Step 4: The laser ranging module directly measures the distance D between the laser (camera) and the ship's waterline. The longitudinal distance between the laser and the waterline can be obtained using the Pythagorean theorem:

[0059]

[0060] Step 5: Combining the calculation formulas from Steps 3 and 4, we can obtain the current image parameters from the camera:

[0061]

[0062] The freeboard height is then:

[0063]

[0064] Example

[0065] A laser rangefinder module (LSP-LRS-0510A) was installed on an 8-megapixel laser gimbal zoom camera (model: iDS-2DY9840IAX-ASP(T5)). The gimbal zoom camera captured a photo with a resolution of 3840×2160, which clearly showed the ship's waterline and deck line.

[0066] Calculate the average value of each row of the image to obtain a 2160-dimensional column vector C, as follows:

[0067] {20,38,89,89,62,67,128,…,158,227} T

[0068] Assuming the width of the ship's waterline and deck line is 5 pixels, we obtain a 2155-dimensional vector Diff1, which looks like this:

[0069] {47,90,-20,57,54,38,…,101,83} T

[0070] Calculate the cumulative value of Diff1 to obtain DiffAcc:

[0071] {47,137,117,174,228,266,…,83101,83184} T

[0072] Performing a difference calculation on DiffAcc yields Diff2:

[0073] {219,283,267,…,158,134} T

[0074] Calculate the two peak values ​​of Diff2, 386 and 372, located in rows 1854 and 1970 respectively, i.e., y1 = 1854, y2 = 1970, yp = 1065, F p =68500;

[0075] The laser measurement distances are D = 584 meters, H = 60 meters, and Y = 580 meters. Finally, Y... g =580*(1970-1854) / 68500 = 0.98 meters Figure 2 This is a rendering of a ship's freeboard identification.

[0076] Based on the same inventive concept, this application provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the aforementioned ship freeboard identification method based on zoom camera and laser ranging.

[0077] Based on the same inventive concept, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the aforementioned ship freeboard identification method based on zoom camera and laser ranging.

[0078] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0079] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0080] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0081] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0082] The above embodiments are merely illustrative of the technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. Any modifications made to the technical solutions based on the technical concept proposed in this invention shall fall within the scope of protection of this invention.

Claims

1. A method for identifying ship freeboard based on zoom camera and laser rangefinder, characterized in that, Includes the following steps: Step 1: Install a zoom gimbal camera on one side of the channel to capture images of the freeboard of ships in the channel; install a laser ranging module on the zoom gimbal camera so that the laser ranging module rotates with the zoom gimbal camera and the distance from the laser ranging module to the ship is the same as the distance from the zoom gimbal camera to the ship. Step 2: Use a zoom gimbal camera to capture an image containing the ship's freeboard. Calculate the vertical edge values ​​of the image containing the ship's freeboard using the cumulative difference method. Identify the two largest peak values ​​from all edge values. Based on these two largest peak values, obtain the ship-water boundary line and deck line, along with their respective row indices in the image containing the ship's freeboard. The specific process is as follows: 1) Define the height (h) and width (w) of the image containing the ship's freeboard captured by the zoom gimbal camera. This represents the grayscale value of the pixel in the i-th row and j-th column of the image. The mean of the gray values ​​in each row of the image is calculated to obtain the column vector C, i.e.: ， in, This represents the average grayscale value of all pixels in the i-th row; 2) Set the width of the ship's waterline and deck line to b pixels each. By performing a difference operation, we obtain : ， in, express The value obtained by difference Indicates the first The mean of the grayscale values ​​of all pixels in the row; 3) Calculation Cumulative value: ， in, express The cumulative value, express The value obtained by difference This represents the average grayscale value of all pixels in the k-th row; 4) Then... By performing a difference operation, we obtain : ， in, express The value obtained by difference express The cumulative value, express The value obtained by difference; 5) From all Find the maximum and second largest values, and find the row indices corresponding to the maximum and second largest values ​​in the image containing the ship's freeboard. Use the smaller of the two row indices as the index value of the ship-water boundary line, and the larger value as the index value of the deck line. Step 3: Calculate the longitudinal distance between the ship's waterline and the zoom gimbal camera based on the transformation model of the zoom gimbal camera and the world coordinate system. The calculation formula is as follows: ， in, H represents the longitudinal distance between the waterline boundary between the boat and the zoom gimbal camera, and H represents the installation height of the laser rangefinder module and the camera. For camera internal parameters, This is the row index for the ship's waterline in an image containing the ship's freeboard. These are the image parameters for the zoom gimbal camera. Step 4: Measure the distance D between the laser ranging module and the waterline of the ship using the laser ranging module. Based on the installation height of the laser ranging module and the Pythagorean theorem, obtain the longitudinal distance between the waterline of the ship and the zoom gimbal camera, i.e.: ， Step 5, based on steps 3 and 4, the calculation formula for the zoom gimbal camera image parameters is as follows: ， Therefore, the ship's freeboard height is: ， in, Indicates the freeboard height of a vessel at sea. This is the row index for the deck line in an image containing the ship's freeboard.

2. The ship freeboard identification method based on zoom camera and laser ranging according to claim 1, characterized in that, The zoom gimbal camera is model iDS-2DY9840IAX-ASP(T5), and the laser rangefinder module is model LSP-LRS-0510A.

3. A computer device, comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the ship freeboard identification method based on zoom camera and laser ranging as described in any one of claims 1 to 2.

4. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the ship freeboard identification method based on zoom camera and laser ranging as described in any one of claims 1 to 2.

Images