Unmanned vessel stereo camera fixed bracket control method for coastline surveying
The method stabilizes stereo camera systems on unmanned vessels by adjusting the fixing bracket in real-time to compensate for hull movements, improving coastline surveying accuracy and reliability.
Patent Information
- Application Number
- KR1020260120297
- Authority / Receiving Office
- KR · KR
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2026-07-01
- Publication Date
- 2026-11-10
- Estimated Expiration
- 2046-07-01
AI Technical Summary
Stereo camera systems on unmanned vessels experience shaking due to roll, pitch, and yaw motions caused by waves, leading to geometric distortion, parallax errors, and reduced reliability in coastline surveying.
A method using an IMU sensor to calculate real-time changes in hull attitude, adjust the stereo camera's fixing bracket to stabilize the shooting posture, and only use images meeting stability criteria for coastline surveying.
Stabilizes coastline surveying by reducing shaking and parallax errors, ensuring reliable and precise coastline variation data acquisition.
Smart Images

Figure 112026079778246-PAT00010_ABST
Abstract
Description
Technology Field
[0001] The present invention relates to a method for controlling a stereo camera fixing bracket of an unmanned vessel for coastline surveying. More specifically, the invention relates to a method for improving the problem in which the shooting reference line and the camera field of view fluctuate due to waves, currents, changes in hull speed, and roll, pitch, and yaw movements of the hull while a stereo camera installed on an unmanned vessel photographs coastlines, embankments, slopes, rocks, structures, and coastal terrain. To this end, the invention calculates the correction angle of the fixing bracket in real time using an IMU sensor, stereo camera images, distance information, and position information, and stabilizes the shooting posture of the stereo camera according to the calculated correction angle, thereby enabling safe and precise photography and surveying of coastline variations even during the operation of the unmanned vessel. Background Technology
[0002] With the recent increase in the need for precise observation of coastline erosion, reclamation, sedimentation, typhoon damage, and changes in coastal structures, there is a demand for technology to photograph and survey coastal areas that are difficult for humans to access directly using unmanned vessels.
[0003] Conventionally, aerial photography, satellite imagery, drone photography, or manual photography by onboard vessels were used for coastline surveying, but these methods had limitations in terms of weather conditions, operating costs, shooting frequency, maritime accessibility, and safety.
[0004] Unmanned vessels have the advantage of being able to photograph the coastline at low altitudes or near the water surface while approaching the coast, allowing for relatively detailed verification of the actual coastline boundaries, water level lines, erosion lines, and structural boundaries.
[0005] However, due to the nature of unmanned vessels operating at sea, roll, pitch, and yaw movements continuously occur caused by waves, swells, currents, and the acceleration and deceleration of the hull itself; these changes in attitude cause the camera's shooting direction to shake, resulting in geometric distortion of the image, focus instability, and parallax errors in the left-right stereo images.
[0006] In particular, for stereo cameras, the reference distance between the left and right cameras, that is, the baseline, must be kept constant, and the optical axis direction of both cameras must be kept stable to ensure the reliability of the 3D distance calculation and coastline survey results.
[0007] While conventional camera devices on unmanned surface vessels may be suitable for attitude stabilization for target detection or surveillance purposes, they were insufficient for fields where geometric reference of captured images, alignment of left and right images, and comparability of iterative survey data are important, such as coastline surveying.
[0008] In addition, when using only a simple gimbal or a manual fixed bracket, the camera's field of view repeatedly shifted toward the coastline or the water surface due to variations in the hull's roll and pitch, resulting in a problem where the quality of continuous shooting for the same coastline section deteriorated.
[0009] Furthermore, if camera shake during filming exceeds a certain level, the image-based coastline extraction algorithm may mistake water surface reflections, wave boundaries, structure shadows, or slope contours for the actual coastline, resulting in reduced reliability of the calculated coastline variation.
[0010] Therefore, a new control method is required that measures the amount of change in hull attitude occurring during the actual operation of an unmanned vessel in real time, actively controls the stereo camera mounting bracket in response to such change, and utilizes only the images from sections where the shooting stability is above a reference value as coastline survey data. Prior art literature
[0011] Republic of Korea Patent Registration No. 10-2671774 (May 29, 2024) 'Attitude Stabilization Target Detection Camera Device for Unmanned Surface Vessels' The problem to be solved
[0012] The objective of the present invention is to provide a method for controlling a stereo camera fixed bracket of an unmanned vessel for coastline surveying, which solves the problem of the optical axis, baseline, and shooting reference angle of a stereo camera shaking due to roll, pitch, and yaw motions caused by waves when an unmanned vessel operates adjacent to a coastline and photographs and surveys coastline variations using a stereo camera; and by calculating the amount of change in hull attitude in real time using an IMU sensor, position information, and stereo image information, and then controlling the correction angle of the fixed bracket, improves the horizontality, overlap, parallax stability, and surveying accuracy of the captured images, and enables the acquisition of coastline variation amounts as quantitative data that can be repeatedly compared. means of solving the problem
[0013] The present invention, as a means to achieve the above-mentioned objective, comprises: a navigation and shooting standard setting step for setting a target separation distance from the coastline, a shooting reference angle of the stereo camera, and an allowable shake reference value; an attitude information collection step for collecting roll, pitch, and yaw attitude information and position information of the unmanned vessel using an IMU sensor and a position sensor equipped on the unmanned vessel; a shooting stability determination step for determining shooting stability using the alignment state of the left and right images of the stereo camera and the attitude information of the unmanned vessel; a bracket correction angle calculation step for calculating the correction direction and correction amount of the stereo camera fixing bracket in response to changes in the roll, pitch, and yaw attitude of the unmanned vessel; a bracket driving step for driving the stereo camera fixing bracket according to the calculated correction direction and correction amount to stabilize the shooting attitude of the stereo camera; and a coastline image shooting step for shooting a coastline image in the stabilized shooting attitude and extracting coastline candidate points from the left and right images. The present invention provides a method for controlling a stereo camera fixing bracket of an unmanned vessel for coastline surveying, characterized by comprising: an image adoption judgment step in which, if the shooting stability is greater than or equal to a reference stability, the image is adopted as an image for coastline surveying, and if the stability is less than the reference stability, the image is excluded or re-shot; and a coastline variation calculation step in which the coastline position is calculated from the adopted image for coastline surveying, and the coastline variation amount is calculated by comparing it with the reference coastline position. Effects of the invention
[0014] According to the present invention, even if an unmanned vessel undergoes roll, pitch, and yaw motions due to waves, the stereo camera fixing bracket can be corrected in real time, thereby reducing shaking and parallax errors in coastal line images.
[0015] In addition, since only images satisfying shooting stability standards can be used for coastline surveying, the reliability of calculating coastline variation can be improved.
[0016] In addition, repetitive and safe coastline surveying can be performed using unmanned vessels even in coastal areas that are difficult for humans to access. Brief explanation of the drawing
[0017] FIG. 1 is a flowchart illustrating a method according to the present invention. Specific details for implementing the invention
[0018] Hereinafter, preferred embodiments according to the present invention will be described in more detail. Prior to describing the present invention, the following specific structural or functional descriptions are merely illustrative examples of embodiments according to the concept of the present invention in various forms and should not be interpreted as being limited to the described embodiments.
[0019] A method for controlling a stereo camera fixed bracket of an unmanned vessel for coastline surveying according to the present invention comprises, as illustrated in the example of FIG. 1, a navigation and shooting standard setting step for setting a target navigation path of the unmanned vessel, a target separation distance from the coastline, a shooting reference angle of the stereo camera, and an allowable shake reference value; an attitude information collection step for collecting roll, pitch, and yaw attitude information and position information of the unmanned vessel using an IMU sensor and a position sensor equipped on the unmanned vessel; a shooting stability determination step for determining shooting stability using the alignment state of the left image and the right image of the stereo camera and the attitude information of the unmanned vessel; a bracket correction angle calculation step for calculating a correction direction and a correction amount of the stereo camera fixed bracket in response to changes in the roll, pitch, and yaw attitude of the unmanned vessel; a bracket driving step for driving the stereo camera fixed bracket according to the calculated correction direction and a correction amount to stabilize the shooting attitude of the stereo camera; and a coastline image shooting step for shooting a coastline image in the stabilized shooting attitude and extracting coastline candidate points from the left image and the right image. It includes: an image adoption judgment step in which, if the shooting stability is greater than or equal to the reference stability, the image is adopted as an image for coastline surveying, and if it is less than the reference stability, the image is excluded or re-shot; and a coastline variation calculation step in which the coastline position is calculated from the adopted image for coastline surveying, and the coastline variation amount is calculated by comparing it with the reference coastline position.
[0020] More specifically, the method for controlling a stereo camera fixing bracket of an unmanned vessel for coastline surveying according to the present invention performs the following steps.
[0021] (1) Stage of setting standards for operation and filming
[0022] In this step, the target navigation path of the unmanned vessel, the target separation distance from the coastline, the stereo camera's shooting cycle, the reference shooting angle, and the allowable shake threshold are set.
[0023] For example, the target separation distance D0 from the coastline can be set to 20m, and the allowable roll angle can be set to ±2°, the allowable pitch angle to ±2°, and the allowable yaw angle to ±3°.
[0024] The technical significance of this stage lies not merely in mounting the camera, but in quantifying the surveying conditions in advance, which serve as criteria for bracket control and the decision on whether to adopt the image.
[0025] (2) Detail information collection stage
[0026] In this step, the roll angle φ(t), pitch angle θ(t), and yaw angle ψ(t) at time t are collected using an IMU sensor, and the position P(t) of the unmanned vessel is collected using GNSS or RTK-GNSS.
[0027] In other words, hull attitude information can be expressed as follows.
[0028] A(t) = {φ(t), θ(t), ψ(t)}
[0029] Here, A(t) is the hull attitude information at time t, φ(t) is the roll angle, θ(t) is the pitch angle, and ψ(t) is the yaw angle.
[0030] For example, if at a specific point in time φ(t)=3.0°, θ(t)=-1.5°, and ψ(t)=2.0°, it means that the hull is tilted 3.0° to the right, the bow direction changes 1.5° downward, and the direction of travel is deflected 2.0°.
[0031] The technical significance of this stage lies in quantifying the actual hull movement caused by sea waves and converting it into an input value for bracket control.
[0032] (3) Shooting stability judgment stage
[0033] In this step, shooting stability is calculated using the number of feature point matches between the left camera image and the right camera image, the amount of parallax change, and the degree of image shake.
[0034] The shooting stability S(t) can be calculated as follows, for example.
[0035] S(t) = 100 - {w₁|ϕ(t)| + w₂|θ(t)| + w₃|ψ(t)| + w₄E(t)}
[0036] Here, S(t) is the image stability, E(t) is the alignment error of the left and right stereo images, and w₁ to w₄ are weights reflecting the importance of each element.
[0037] In this case, since roll and pitch have a significant impact on image horizontality and optical axis maintenance in coastline surveying, the weights can be set, for example, w₁=6, w₂=5, w₃=3, and w₄=2.
[0038] As an actual calculation example, if φ(t)=3.0°, θ(t)=1.5°, ψ(t)=2.0°, and E(t)=4, then S(t)=100-(6×3.0+5×1.5+3×2.0+2×4)=100-(18+7.5+6+8)=60.5.
[0039] If the reference stability is set to 70, the calculated value of 60.5 is lower than the reference value, so it means that the image at that point in time cannot be used as is for surveying and requires bracket correction or re-capturing of the image.
[0040] The technical significance of this stage lies in determining the usability of the captured footage based on numerical criteria rather than judging it sensorially.
[0041] (4) Step to calculate bracket correction angle
[0042] In this step, a correction angle is calculated to drive the stereo camera mounting bracket in a direction opposite to the roll, pitch, and yaw changes of the hull.
[0043] At this time, the correction angle C(t) can be expressed as follows.
[0044] C(t) = {-kψ·ψ(t), -kθ·θ(t), -kψ·ψ(t)}
[0045] Here, C(t) is the set of roll correction angle, pitch correction angle, and yaw correction angle of the fixed bracket, and kφ, kθ, and kψ are the roll, pitch, and yaw correction coefficients, respectively.
[0046] In this case, the correction factor can be set in the range of 0.7 to 1.2 depending on the responsiveness of the bracket motor, the size of the hull, the weight of the camera, and the sea conditions, and is basically set to close to 1.0 in the precision survey mode.
[0047] For example, if φ(t)=3.0°, θ(t)=-1.5°, ψ(t)=2.0°, and kφ=1.0, kθ=0.9, kψ=0.8, then C(t)={-3.0°, +1.35°, -1.6°}.
[0048] This means that the bracket is driven in the opposite direction to compensate for the hull being tilted 3.0° to the right, tilted -1.5° forward, and the direction of travel being turned 2.0°.
[0049] The technical significance of this step is to maintain the coastline shooting standard by compensating for changes in hull attitude caused by waves in the camera shooting coordinate system.
[0050] (5) Bracket driving stage
[0051] In this step, at least one drive motor among the roll axis, pitch axis, and yaw axis is controlled to ensure that the optical axis of the stereo camera maintains the shooting direction of the target coastline.
[0052] At this time, to prevent image shaking caused by rapid motor operation, the amount of change between the current correction angle and the previous correction angle can be limited.
[0053] For example, if the maximum correction angle change amount per control cycle is limited to 1.0°, even if a roll correction value of -3.0° is calculated instantaneously, it can be controlled stepwise in the order of -1.0°, -2.0°, and -3.0°.
[0054] The technical significance of this step lies not in simple shake correction, but in simultaneously stabilizing the left and right image reference lines of the stereo camera and the direction of the coastline shooting.
[0055] (6) Coastline video recording stage
[0056] In this step, left and right images are captured simultaneously in a corrected bracket position, and coastline candidate points are extracted by analyzing water surface boundaries, soil slope boundaries, structure boundaries, vegetation boundaries, and brightness change lines within the images.
[0057] At this time, the coastline candidate point can be represented by image coordinates (u, v), and the actual distance Z can be calculated using stereo parallax d.
[0058] Z = f·B / d
[0059] Here, Z is the distance from the camera to the target point, f is the focal length, B is the baseline between the left and right cameras, and d is the parallax between identical feature points in the left and right images.
[0060] For example, if f=12mm, B=0.5m, and d=0.003m, then Z=(0.012×0.5) / 0.003=2m.
[0061] In the actual system, pixel-unit parallax is converted into length units based on camera intrinsic parameters and calculated.
[0062] The technical significance of this step is that it enables the calculation of the 3D location of coastline candidate points using stabilized stereo images.
[0063] (7) Video acceptance decision stage
[0064] In this step, the previously calculated shooting stability S(t) is compared with the reference stability S0.
[0065] For example, if S0 is set to 70, if S(t) ≥ 70, the corresponding image frame is adopted as the image for surveying, and if S(t) < 70, the corresponding image frame is excluded or a re-capture command is generated.
[0066] In the previous calculation example, S(t)=60.5, which is lower than the reference value of 70, so re-shooting or correction reinforcement is required.
[0067] Conversely, if φ(t)=0.8°, θ(t)=0.5°, ψ(t)=0.7°, and E(t)=2 after bracket control, S(t)=100-(6×0.8+5×0.5+3×0.7+2×2)=86.6, which is higher than the reference value of 70, so it can be adopted as a survey image.
[0068] The technical significance of this step is that it automatically selects the quality of coastline survey data by considering control values and image matching values together, rather than arbitrarily judging the quality after shooting.
[0069] (8) Step to calculate coastline variation
[0070] In this step, coastline coordinates are calculated from the adopted stereo image, and the coastline variation amount ΔL is calculated by comparing it with the coastline coordinates from the previous survey point.
[0071] ΔL = L n - L0
[0072] Here, ΔL is the coastline variation, L n represents the current surveyed coastline location, and L0 represents the coastline location at the reference point.
[0073] For example, if the reference coastline position is 15.2m from the reference coordinates and the current coastline position is 13.8m, then ΔL=13.8-15.2=-1.4m, so it can be interpreted that the coastline has retreated 1.4m in the direction of the reference line.
[0074] The technical significance of this stage is that it enables the quantitative determination of whether coastline erosion or sedimentation is occurring using stabilized imaging results.
[0075] In addition, a camera stability index calculation step can be performed.
[0076] In this step, the Camera Stability Index is calculated to quantitatively evaluate the shooting stability of the stereo camera.
[0077] The camera stability index Rc can be calculated as follows.
[0078]
[0079]
[0080] At this time, weight And, generally, since roll and pitch have a significant impact on image distortion in coastline surveying,
[0081]
[0082] It is desirable to set it as such.
[0083] Accordingly, the posture value measured at a certain point in time,
[0084]
[0085] ramen,
[0086]
[0087] It becomes.
[0088] Here, the stability index is defined as follows.
[0089]
[0090] because,
[0091] silver Therefore, the system saves the current video but determines that it is in a "re-shoot required" state.
[0092] If Rc > 4.0, it is determined that the hull shaking caused by waves is excessive, and at least one of stopping filming, recalibrating the bracket, reducing the ship's speed, or waiting for stabilization can be performed.
[0093] Accordingly, the present invention improves the problem in which the shooting reference line and camera field of view fluctuate due to waves, currents, changes in hull speed, and roll, pitch, and yaw movements of the hull while a stereo camera installed on an unmanned vessel is shooting coastlines, embankments, slopes, rocks, structures, and coastal terrain by calculating the correction angle of the fixed bracket in real time using an IMU sensor, stereo camera images, distance information, and position information, and by stabilizing the shooting posture of the stereo camera according to the calculated correction angle, thereby enabling safe and precise shooting and surveying of coastline variations even during the operation of the unmanned vessel.
Claims
Claim 1 delete Claim 2 A navigation and shooting standard setting step for setting a target separation distance from the coastline, a shooting reference angle of the stereo camera, and an allowable shake reference value; an attitude information collection step for collecting roll, pitch, and yaw attitude information and position information of the unmanned vessel using an IMU sensor and a position sensor equipped on the unmanned vessel; a shooting stability determination step for determining shooting stability using the alignment status of the left and right images of the stereo camera and the attitude information of the unmanned vessel; a bracket correction angle calculation step for calculating the correction direction and correction amount of the stereo camera fixing bracket in response to changes in the roll, pitch, and yaw attitude of the unmanned vessel; a bracket driving step for driving the stereo camera fixing bracket according to the calculated correction direction and correction amount to stabilize the shooting attitude of the stereo camera; a coastline image shooting step for capturing a coastline image in the stabilized shooting attitude and extracting coastline candidate points from the left and right images; and an image adoption determination step for adopting the image as a coastline survey image if the shooting stability is greater than or equal to the reference stability, and excluding or reshooting the image if it is less than the reference stability. A method for controlling a stereo camera fixed bracket of an unmanned vessel for coastline surveying, characterized by including: a step for calculating a coastline variation amount, which calculates a coastline position from the adopted coastline surveying image and calculates a coastline variation amount by comparing it with a reference coastline position; wherein the step for calculating a bracket correction value is characterized by calculating the correction angle C(t) of the stereo camera fixed bracket using the following formula, where the roll angle at time t is φ(t), the pitch angle is θ(t), the yaw angle is ψ(t), the roll correction coefficient is kφ, the pitch correction coefficient is kθ, and the yaw correction coefficient is kψ. C(t) = {-kφ·φ(t), -kθ·θ(t), -kψ·ψ(t)}
Citation Information
Patent Citations
Drawing system of an aerial photograph
KR101252680B1
System for constructing and browsing geographicinformation using video data and method thereof
KR1020040024624A
Apparatus for extracting shoreline and method thereof
KR1020260000502A
