Draft measurement system and method based on image processing using artificial intelligence technology

Abstract

The present invention proposes a draft measurement system and method based on image processing using artificial intelligence technology, which applies artificial intelligence technology to images extracted from data, thereby preventing image processing errors that occur due to the shooting angle, image quality, and the state of contamination of the hull surface, thereby improving the accuracy of draft measurement, and which uses technologies such as a deep learning model for detecting the edges of draft marks and performing the segmentation required to identify the waterline surface, as well as techniques for specifying a reference draft point and a distortion correction algorithm to confirm the draft value based on video data captured in real time.

Description

[Technical field]

[0001] The present invention relates to a draft measurement system and a draft measurement method based on image processing using artificial intelligence technology, which applies artificial intelligence technology to images extracted from video data, prevents image processing errors caused by the shooting angle, image quality, the contamination state of the hull surface, and the like, improves the accuracy of draft measurement, and uses technologies such as a deep learning model for detecting the edges of draft marks and performing segmentation required for identifying the waterline surface, an edge detection algorithm for detecting the waterline surface, and a distortion correction algorithm to confirm the draft value based on video data captured in real time. [Background technology]

[0002] Normally, a ship undergoes a list test and a dead-weight test (dead-weight measurement) during construction and at the launching stage after completion of construction. At this time, it is necessary to measure the ship's draft in order to calculate the displacement below the ship's waterline. Conventionally, draft measurements are performed by a trained measurer who boards a work boat and approaches draft marks marked on the side of the ship at the bow, center, and stern, and visually measures using a draft gauge. However, such measurement work has the risk of causing accidents such as contusions, drowning, and musculoskeletal injuries due to the unstable posture of the measurer, and there is also the problem that the measured values ​​vary depending on the environmental conditions such as wind, waves, and swell, and the skill level of the measurer. Summary of the Invention [Problem to be solved by the invention]

[0003] The object of the present invention is to provide a draft measurement system and method based on image processing applying artificial intelligence technology, which applies artificial intelligence technology to images extracted from video data to prevent image processing errors caused by the shooting angle, image quality, the state of contamination of the hull surface, etc., thereby improving the accuracy of draft measurement, and which can confirm the draft value based on video data captured in real time using a deep learning model for performing the segmentation required to detect the edges of draft marks and identify the waterline surface, an edge detection algorithm and a distortion correction algorithm for detecting the waterline surface, as well as a drone and a small video camera, video reception / transmission and streaming technology.

[0004] Another object of the present invention is to provide a draft measurement system and method based on image processing using artificial intelligence technology, which applies artificial intelligence technology to images extracted from video data to prevent image processing errors caused by the shooting angle, image quality, the state of contamination of the hull surface, etc., thereby improving the accuracy of draft measurement, and which can confirm the draft value using technologies such as a deep learning model for detecting the edges of draft marks and performing the segmentation required to identify the waterline surface, designating a reference draft point, and a distortion correction algorithm. [Means for solving the problem]

[0005] In order to solve the above problems, a draft measurement system based on image processing to which artificial intelligence technology of an embodiment of the present invention is applied includes at least one of a drone and a video capture device that captures a draft mark including a number marked on the outer side of a ship and a waterline surface, and a draft measurement system that receives images captured by at least one of the drone and the video capture device and calculates the draft. The draft measurement system includes a pre-processing unit that converts the received images into images in reference frame units, a draft mark identification unit that identifies a draft mark from the image converted into reference frame units by the pre-processing unit, a waterline detection unit that detects a waterline from the image converted into reference frame units by the pre-processing unit, a measurement unit that measures the pixel distance between the draft mark identified by the draft mark identification unit and the waterline detected by the waterline detection unit, a distortion correction unit that corrects distortion in the X-axis direction and distortion in the Y-axis direction of the measurement data measured by the measurement unit, and a calculation unit that calculates the draft based on the corrected data value corrected by the distortion correction unit.

[0006] In addition, the waterline detection unit of the draft measurement system according to an embodiment of the present invention further includes an edge detection unit that detects edges of the waterline from the image converted into reference frame units by the pre-processing unit, and identifies the waterline surface by segmentation based on the edges of the waterline detected by the edge detection unit.

[0007] In addition, the measurement unit of the draft measurement system of an embodiment of the present invention is characterized by having a first measurement unit that measures the pixel distance between a draft mark identified by the draft mark identification unit and a draft mark above the waterline, and a second measurement unit that measures the pixel distance between the draft mark identified by the draft mark identification unit and the waterline.

[0008] In addition, the distortion correction unit of the draft measurement system of an embodiment of the present invention is characterized in that it performs a perspective transformation in which the surface of a ship with draft marks is regarded as a plane, measures the height of the waterline from the image corrected by this perspective transformation, corrects the distortion in the X-axis direction, predicts a value in the Y-axis direction based on the correction of the distortion in the X-axis direction, and corrects the distortion in the Y-axis direction.

[0009] In addition, the draft measurement system of an embodiment of the present invention is characterized in that it further includes a monitoring server that at least one of displays and stores the draft value in real time in order to monitor the draft value calculated by the calculation unit.

[0010] A draft measurement system based on image processing to which artificial intelligence technology according to an embodiment of the present invention is applied includes at least one of a drone and a video capture device that captures images of a waterline and a draft mark including numbers marked on the outer side of a ship, and a draft measurement system that receives images captured by at least one of the drone and the video capture device and calculates the draft. The draft measurement system includes a pre-processing unit that converts the received images into an image in units of reference frames, and detects draft marks as objects from the images converted into units of reference frames by the pre-processing unit, and calculates the draft. The apparatus is characterized in that it comprises an object identification unit that identifies a position, a draft measurement reference point definition unit that detects a waterline from an image converted into reference frame units by the preprocessing unit and defines a reference point for draft measurement, a distortion correction unit that corrects distortion of the draft mark identified by the object identification unit and distortion of the draft measurement reference point defined by the draft measurement reference point definition unit, a measurement unit that measures the distance between the draft mark whose distortion has been corrected by the distortion correction unit and the draft measurement reference point whose distortion has been corrected by the distortion correction unit, and a calculation unit that calculates the draft based on the measurement data value measured by the measurement unit.

[0011] In addition, a draft measurement method based on image processing to which artificial intelligence technology of an embodiment of the present invention is applied includes a video capture step of capturing images of a draft mark including a number marked on the outer side of a ship and a waterline surface by at least one of a drone and a video capture device; a pre-processing step of receiving the image captured in the video capture step at a draft measurement system and converting the received image into an image in reference frame units; a draft mark identification step of identifying a draft mark from the image converted into reference frame units in the pre-processing step; a waterline detection step of detecting a waterline from the image converted into reference frame units in the pre-processing step; a measurement step of measuring the pixel distance between the draft mark identified in the draft mark identification step and the waterline detected in the waterline detection step; a distortion correction step of correcting distortion in the X-axis direction and distortion in the Y-axis direction of the measurement data measured in the measurement step; and a calculation step of calculating the draft based on the correction data corrected in the distortion correction step.

[0012] Moreover, the waterline detection step of the draft measurement method of the embodiment of the present invention is characterized in that it includes an edge detection step of detecting the edges of the waterline in order to detect the waterline, and a step of identifying the waterline surface by segmentation based on the edges of the detected waterline.

[0013] Moreover, the measurement step of the draft measurement method of the embodiment of the present invention is characterized by including a first measurement step of measuring the pixel distance between the draft mark identified in the draft mark identification step and a draft mark above the waterline, and a second measurement step of measuring the pixel distance between the draft mark identified in the draft mark identification step and the waterline.

[0014] Furthermore, the distortion correction step of the draft measurement method of the embodiment of the present invention is characterized by including a perspective transformation step of regarding the surface of the ship on which the draft marks are made as a plane, a step of measuring the height of the waterline from the image corrected in the perspective transformation step and correcting the distortion in the X-axis direction, and a step of predicting a value in the Y-axis direction based on the correction of the distortion in the X-axis direction in the step of correcting the distortion in the X-axis direction, and correcting the distortion in the Y-axis direction.

[0015] In addition, the draft measurement method of an embodiment of the present invention is characterized in that it further includes a monitoring step of transmitting the draft value calculated in the calculation step to a monitoring server and at least one of displaying it in real time and storing it, in order to monitor the draft value calculated in the calculation step.

[0016] Further, a draft measurement method based on image processing to which artificial intelligence technology of an embodiment of the present invention is applied includes a video capture step of capturing images of a draft mark including numbers marked on the outer side of a ship and a waterline surface by a video capture device, a pre-processing step of receiving the image captured in the video capture step by a draft measurement system and converting the received image into an image in reference frame units in the pre-processing step, an object identification step of detecting the draft mark as an object from the image converted into reference frame units in the pre-processing step and identifying its position, and a waterline detection step of detecting the draft mark from the image converted into reference frame units in the pre-processing step, The method includes a draft measurement reference point definition step of defining a reference point for draft measurement, a distortion correction step of correcting distortion of the draft mark identified in the object identification step and distortion of the draft measurement reference point defined in the draft measurement reference point definition step, a measurement step of measuring a distance between the draft mark whose distortion has been corrected in the distortion correction step and the draft measurement reference point whose distortion has been corrected in the distortion correction step, an outlier removal step of removing outliers from data values ​​measured in the measurement step, and a calculation step of calculating the draft based on the data from which the outliers have been removed in the outlier removal step.

[0017] In addition, the object identification step of the draft measurement method of the embodiment of the present invention is characterized in that a Mask R-CNN model is applied to detect draft marks as objects from images converted into reference frame units and identify their positions.

[0018] In addition, the draft measurement reference point definition step of the draft measurement method of an embodiment of the present invention is characterized in that in order to detect the waterline, an artificial intelligence model including U-Net is applied to perform semantic segmentation to identify the position of the edge of the waterline, and the intersection of a straight line selected by applying a voting algorithm based on the object detected in the object identification step and the semantic segmented waterline and the semantic segmented waterline is defined as the reference point for draft measurement.

[0019] In addition, the distortion correction step of the draft measurement method of the embodiment of the present invention is characterized in that a scaling algorithm is applied to correct distortion of the size of the draft mark due to the shooting position, and the scaling algorithm estimates the height of the draft mark based on a proportional change in the size of the draft mark across the entire image in frame units to correct the distortion.

[0020] In addition, the outlier removal step of the draft measurement method of an embodiment of the present invention is characterized in that outliers are detected and removed from the results of analyzing each image frame by frame based on statistical univariate data of standard scores or interquartile ranges, and the draft value is derived as an arithmetic mean.

[0021] In addition, the draft measurement method of an embodiment of the present invention is characterized in that it further includes a monitoring step of transmitting the draft value calculated in the calculation step to a monitoring server and at least one of displaying it in real time and storing it, in order to monitor the draft value calculated in the calculation step.

[0022] Furthermore, in order to achieve the above object, a computer-readable recording medium according to an embodiment of the present invention stores a computer program for causing a computer to execute the draft measurement method. Effect of the Invention

[0023] The present invention applies artificial intelligence technology to images extracted from video data, preventing image processing errors caused by factors such as the shooting angle, image quality, and contamination state of the hull surface, thereby improving the accuracy of draft measurement.

[0024] In addition, the present invention can confirm the draft value based on video data captured in real time using a deep learning model for detecting the edges of draft marks and performing the segmentation required to identify the waterline surface, an edge detection algorithm and a distortion correction algorithm for detecting the waterline surface, as well as a drone and a small video camera, video reception / transmission and streaming technology.

[0025] The present invention also prevents accidents such as contusions, drowning, and musculoskeletal injuries that may occur when measuring draft, and provides more reliable and accurate measurements than visual measurements. [Brief description of the drawings]

[0026] [Figure 1] FIG. 1 is a diagram showing a configuration of a draft measurement system according to a first embodiment of the present invention. [Diagram 2] FIG. 2 is a flowchart showing a draft measurement method according to the first embodiment of the present invention. [Diagram 3] 4A to 4C are diagrams illustrating an image capturing step of the draft measuring method according to the first embodiment of the present invention. [Figure 4] FIG. 2 is a diagram illustrating an extraction process of the draft measurement method according to the first embodiment of the present invention. [Diagram 5] FIG. 2 is a diagram illustrating a waterline detection step of the draft measurement method according to the first embodiment of the present invention. [Figure 6]FIG. 2 is a conceptual diagram for explaining edge detection and waterline detection by segmentation in the draft measurement method according to the first embodiment of the present invention. [Figure 7] FIG. 2 is a conceptual diagram for explaining edge detection and waterline detection by segmentation in the draft measurement method according to the first embodiment of the present invention. [Figure 8] FIG. 2 is a diagram for explaining a process of correcting distortion and calculating a waterline in the draft measurement method according to the first embodiment of the present invention. [Figure 9] 4A to 4C are diagrams illustrating character identification and post-processing steps of a draft mark in the draft measurement method according to the first embodiment of the present invention. [Figure 10] FIG. 11 is a flowchart of a draft measurement method according to a second embodiment of the present invention. [Figure 11] FIG. 11 is a diagram for explaining an image recognition and calculation process by a draft calculation algorithm using the draft measurement system according to the second embodiment of the present invention. [Figure 12] FIG. 11 is a diagram for explaining an algorithm for estimating the position of a reference point, among the draft calculation algorithms using the draft measurement system according to the second embodiment of the present invention. [Figure 13] FIG. 11 is a diagram showing a result of applying a voting algorithm after a draft calculation algorithm using the draft measurement system according to the second embodiment of the present invention. [Figure 14] FIG. 11 is a diagram for explaining a scaling algorithm among the draft calculation algorithms using the draft measurement system according to the second embodiment of the present invention. [Figure 15] FIG. 11 is a diagram for explaining a scaling algorithm among the draft calculation algorithms using the draft measurement system according to the second embodiment of the present invention. [Figure 16] FIG. 11 is a diagram for explaining an algorithm for estimating the height of characters, among the draft calculation algorithms using the draft measurement system according to the second embodiment of the present invention. [Figure 17] FIG. 11 is a diagram for explaining an abnormal value criterion application algorithm for deriving a final draft value, among the draft calculation algorithms using the draft measurement system according to the second embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0027] The details of the objectives, technical configuration, operation and effects of the present invention will be more clearly understood from the detailed description based on the drawings attached to the specification of the present invention.

[0028] The terms used in this specification are used to describe certain embodiments and are not intended to limit the present invention. For example, the terms "comprise", "include", "include" and the like used in this specification are not necessarily interpreted as including all of the components or steps described in the present specification, but may include some components or steps, or may further include additional components or steps. Furthermore, the singular expressions used in this specification include the plural expressions unless otherwise clearly understood in the context.

[0029] Hereinafter, the present invention will be described in detail with reference to the accompanying drawings. The embodiments described below are provided for the purpose of allowing those skilled in the art to easily understand the technical concept of the present invention, and the present invention is not limited to the embodiments. It is obvious that the embodiments of the present invention can be applied in various ways to those skilled in the art.

[0030] 1 to 9, the draft measurement system based on image processing to which artificial intelligence technology of the first embodiment of the present invention is applied includes a drone 100 or an imaging device 110 such as a camcorder that captures video of draft marks including numbers marked on the waterline surface and the outside of both sides of a ship, as shown in FIG. 1, and a draft measurement system 200 that receives the video (video) captured by the imaging device 110 such as the drone 100 or a camcorder and calculates the draft based on the video.

[0031] The draft measurement system 200 of this embodiment includes a pre-processing unit 210 that converts the received video into an image in reference frame units, a draft mark identification unit 220 that identifies draft marks from the image converted into reference frame units by the pre-processing unit 210, a waterline detection unit 240 that detects a waterline from the image converted into reference frame units by the pre-processing unit 210, a measurement unit 250 that measures the distance in pixels from the draft mark identified by the draft mark identification unit 220 to the waterline detected by the waterline detection unit 240, a distortion correction unit 260 that corrects distortion in the X-axis and Y-axis directions of the measurement data measured by the measurement unit 250, and a calculation unit 270 that calculates the value of the draft based on the corrected data corrected by the distortion correction unit 260.

[0032] In this embodiment, the waterline detection unit 240 further includes an edge detection unit 230 that detects edges of waterlines from the images converted into reference frame units. The edge detection unit 230 identifies the waterline surface by segmentation based on the edges of the detected waterlines.

[0033] In addition, in this embodiment, the measurement unit 250 is equipped with a first measurement unit 251 that measures the pixel distance from the draft mark identified by the draft mark identification unit 220 to a draft mark above the waterline, and a second measurement unit 252 that measures the pixel distance from the draft mark identified by the draft mark identification unit 220 to the waterline.

[0034] In addition, in this embodiment, the distortion correction unit 260 performs a perspective transform in which the surface of the ship with draft marks is regarded as a plane, and corrects the distortion in the X-axis direction by measuring the height of the waterline from the image corrected by the perspective transform, and also corrects the distortion in the Y-axis direction by predicting the value in the Y-axis direction based on the distortion correction in the X-axis direction.

[0035] In addition, in this embodiment, in order to monitor the value of the draft calculated by the calculation unit 270, a monitoring server 300 is provided that displays or stores the calculated value of the draft in real time.

[0036] That is, the monitoring server 300 includes a storage unit 310 that stores the calculated draft value so that the calculated draft value can be monitored in real time, and a display unit 320 that displays the calculated draft value.

[0037] Also, referring to FIG. 2, the draft measurement method of this embodiment includes an image capturing step S100 in which a drone 100, a video capture device 110 such as a camcorder, captures images of draft marks including numbers marked on the outside of the side of a ship and the water line surface, a pre-processing step S110 in which an image captured by the drone 100, a camcorder, or other video capture device 110 in the image capturing step S100 is received and the received image is converted into an image in units of reference frames by a pre-processing step 210 of the draft measurement system 200, and a draft mark identification step S110 in which the draft measurement system 200 identifies draft marks from the image converted into units of reference frames in the pre-processing step S110. 20, a waterline detection step S140 in which the draft measurement system 200 detects a waterline from the image converted into reference frame units in the preprocessing step S110, a measurement step S150 in which the pixel distance from the draft mark identified in the draft mark identification step S120 to the waterline detected in the waterline detection step S140 is measured, a distortion correction step S160 in which the draft measurement system 200 corrects distortion in the X-axis direction and the Y-axis direction of the measurement data measured in the measurement step S150, and a calculation step S170 in which the draft measurement system 200 calculates a draft value based on the corrected data corrected in the distortion correction step S160.

[0038] In the draft measurement method of the present embodiment, the waterline detection step S140 further includes an edge detection step S130 for detecting the edge of the waterline for the draft measurement system 200. The waterline surface is identified by segmentation based on the edge of the waterline detected in the edge detection step S130.

[0039] In addition, in the draft measurement method of this embodiment, the measurement step S150 includes a first measurement step S151 in which a deep learning model part of the draft measurement system 200 measures the pixel distance from the draft mark identified in the draft mark identification step S120 to a draft mark above the waterline, and a second measurement step S152 in which a deep learning model part of the draft measurement system 200 measures the pixel distance from the draft mark identified in the draft mark identification step S120 to the waterline.

[0040] Moreover, in the draft measurement method of this embodiment, the distortion correction step S160 includes a perspective transformation step S161 in which the surface of the ship with draft marks is regarded as a plane, an X-axis direction distortion correction step S162 in which the height of the waterline is measured from the image corrected in the perspective transformation step S161 and distortion in the X-axis direction is corrected, and a Y-axis direction distortion correction step S163 in which a value in the Y-axis direction is predicted by the X-axis direction distortion correction step S161 and distortion in the Y-axis direction is corrected.

[0041] In addition, the draft measurement method of this embodiment further includes a monitoring step S180 in which the draft value calculated in the calculation step S170 is transmitted to a monitoring server 300 for monitoring, and the draft value is displayed or stored in real time.

[0042] FIG. 3 is a diagram illustrating the image capturing step S100 of the draft measurement method of this embodiment, and shows the state in which the flight of the drone 100 is controlled to capture an image of the draft marks, including numbers marked on the outside of the side of the ship, and the waterline surface using the drone 100.

[0043] Alternatively, a video capture device 110 such as a camcorder may be used to capture images of the draft marks, including the numbers marked on the outside of the side of the vessel, and the waterline surface.

[0044] At this time, the images captured by the drone 100 or the video capture device 110 are transmitted as video (video) data via the network 50 to the draft measurement system 200, and the draft measurement system 200 performs draft measurement based on the received video (video) data.

[0045] When the draft measurement system 200 of the present embodiment receives the video data, it executes video multi-streaming. At this time, the received video data may be immediately transmitted to the monitoring server 300, or the video data may be transmitted to the monitoring server 300 after the draft measurement.

[0046] Fig. 4 is a diagram for explaining the extraction steps of the draft measurement method of this embodiment. Fig. 4(a) shows the process of extracting the waterline from the image converted into the reference frame unit in the pre-processing step S110. Fig. 4(b) shows the process of extracting the character area and geometric information of the draft mark from the image converted into the reference frame unit in the pre-processing step S110.

[0047] FIG. 4(c) shows the process of aggregating the extracted information to extract regions of interest.

[0048] That is, by measuring the pixel distance from the reference draft mark to a draft mark above the waterline, and then measuring the pixel distance from the reference draft mark to the waterline, an accurate value for the waterline can be obtained.

[0049] In addition, the waterline is read by using the Mask R-CNN model to identify the reference mark M area for reading, and reading it based on the distance ratio between this reference mark and the surrounding number marks, while also identifying the sea surface position.

[0050] That is, by measuring the pixel distance from the reference draft mark to a draft mark above the waterline, and then measuring the pixel distance from the reference draft mark to the waterline, an accurate value for the waterline can be obtained.

[0051] More specifically, when extracting a draft mark, a first label image may be formed by assigning a first label to a number area of ​​the draft mark, and a second label image may be formed by assigning a second label to an area between the numbers of the draft mark in the first label image.

[0052] In addition, when forming the first label image, the number area of ​​the draft mark may be identified, and the image of the number corresponding thereto may be identified, and then the first label may be assigned to this area as a first mark of a predetermined size to form the first label image. In addition, when forming the second label image, the area between the numbers displayed in the first label image may be identified, and the image of the number corresponding to the intermediate value existing between these numbers may be identified, and then the second label may be assigned to this area as a second mark of a predetermined size to form the second label image.

[0053] In addition, the draft measurement system 200 may perform identification of the number area of ​​the draft mark and the corresponding number image when forming the first label image, and may perform calculation of the area between the numbers and the numbers corresponding to the intermediate values ​​existing between the numbers when forming the second label image.

[0054] Furthermore, when extracting a draft mark, the draft measurement system 200 may calculate the draft using a numerical ratio between a first label adjacent to the waterline and a second label adjacent to the first label based on a standard size of the draft mark. Alternatively, the draft mark may be extracted using a numerical ratio between a second label adjacent to the waterline and a first label adjacent to the second label based on a standard size of the draft mark.

[0055] Fig. 5 is a diagram for explaining the waterline detection step of the draft measurement method of this embodiment. Fig. 5(a) shows an original image converted from video data captured by a video capture device 110 such as a drone 100 or a camcorder. Fig. 5(b) shows a process of detecting the waterline by applying an artificial intelligence model of the draft measurement system. Fig. 5(c) shows a state in which the waterline surface is identified by segmentation of the waterline.

[0056] That is, the draft measurement system 200 can apply the U-Net algorithm to identify the waterline surface by segmenting the waterline.

[0057] 6 and 7 are conceptual diagrams for explaining the edge detection step and the waterline detection step by segmentation in the draft measurement method of this embodiment. Fig. 6(a) shows the original image, Fig. 6(b) shows the edge-detected image, and Fig. 6(c) shows the inference result.

[0058] Moreover, Fig. 7(a) shows the original image, Fig. 7(b) shows the segmentation image, and Fig. 7(c) shows the detection result.

[0059] This makes it possible to detect the edge of the waterline, and the waterline can be detected based on the detected edge of the waterline.

[0060] 8 is a diagram for explaining the process of distortion correction and waterline derivation in the draft measurement method of this embodiment. The surface of the ship with the draft marks is regarded as a plane by perspective transformation using character area information, and the height of the waterline is measured by distortion correction in the X-axis direction of the image corrected by perspective transformation, and the value of the waterline can be calculated by distortion correction in the Y-axis direction.

[0061] This allows the waterline to be measured more accurately and quickly than by visual measurement.

[0062] Fig. 9 is a diagram for explaining the character identification and post-processing process of the draft mark in the draft measurement method of the present embodiment. Fig. 9 shows the process of identifying the characters of the draft mark, extracting the draft mark by post-processing to correct distortion in the X-axis direction and distortion in the Y-axis direction, and measuring the pixel distance from the extracted draft mark to the waterline.

[0063] Next, a draft measurement system based on image processing to which artificial intelligence technology is applied according to a second embodiment of the present invention and a draft measurement method using this draft measurement system will be described with reference to Figures 10 to 17. As in the first embodiment, the draft measurement system of this embodiment includes a drone 100 or an image capture device 110 such as a camcorder that captures a video of draft marks including numbers marked on the waterline and the outside of both sides of a ship, and a draft measurement system 200 that receives the video (video) captured by the drone 100 or the image capture device 110 such as a camcorder and calculates the draft based on the video.

[0064] The draft measurement system 200 of this embodiment includes a pre-processing unit 210 that converts the received video into an image in reference frame units, an object identification unit 220 that detects draft marks as objects from the image converted into reference frame units by the pre-processing unit 210 and specifies (identifies) their positions, a draft measurement reference point definition unit 240 that detects waterlines from the image converted into reference frame units by the pre-processing unit 210 and defines reference points for draft measurement, a distortion correction unit 250 that corrects distortion of the draft marks identified by the object identification unit 220 and distortion of the draft measurement reference points defined by the draft measurement reference point definition unit 240, a measurement unit 260 that measures the distance from the draft marks whose distortion has been corrected by the distortion correction unit 250 to the draft measurement reference points whose distortion has been corrected by the distortion correction unit 250, and a calculation unit 270 that calculates the value of the draft based on the measurement data measured by the measurement unit 260.

[0065] In this embodiment, the draft measurement reference point definition unit 240 further includes an edge detection unit 230 that detects the edge of the waterline from the image converted into the reference frame unit. The edge detection unit 230 identifies the waterline surface by semantic segmentation based on the edge of the detected waterline.

[0066] In addition, in this embodiment, the measurement unit 260 includes a first measurement unit 261 that measures the pixel distance from the draft mark identified by the object identification unit 220 to a draft mark above the waterline, and a second measurement unit 262 that measures the pixel distance from the draft mark identified by the object identification unit 220 to the waterline.

[0067] In addition, a drone 100 or an imaging device 110 such as a camcorder captures images of the draft marks, including the numbers marked on the outside of the side of the vessel, and the waterline surface.

[0068] At this time, the images captured by the drone 100 or the video capture device 110 are transmitted as video (video) data via the network 50 to the draft measurement system 200, and the draft measurement system 200 performs draft measurement based on the received video (video) data.

[0069] When the draft measurement system 200 of the present embodiment receives the video data, it executes video multi-streaming. At this time, the received video data may be immediately transmitted to the monitoring server 300, or the video data may be transmitted to the monitoring server 300 after the draft measurement.

[0070] Furthermore, in the draft measurement system 200 of this embodiment, the object identification unit 220 applies an artificial intelligence model including Mask R-CNN to identify objects.

[0071] In addition, the waterline is read by using the Mask R-CNN model to identify the reference mark M area for reading, and reading it based on the distance ratio between this reference mark and the surrounding number marks, while also identifying the sea surface position.

[0072] That is, by measuring the pixel distance from the reference draft mark to a draft mark above the waterline, and then measuring the pixel distance from the reference draft mark to the waterline, an accurate value for the waterline can be obtained.

[0073] More specifically, when extracting a draft mark, a first label image may be formed by assigning a first label to a number area of ​​the draft mark, and a second label image may be formed by assigning a second label to an area between the numbers of the draft mark in the first label image.

[0074] In addition, when forming the first label image, the number area of ​​the draft mark may be identified, and the image of the number corresponding thereto may be identified, and then the first label may be assigned to this area as a first mark of a predetermined size to form the first label image. In addition, when forming the second label image, the area between the numbers displayed in the first label image may be identified, and the image of the number corresponding to the intermediate value existing between these numbers may be identified, and then the second label may be assigned to this area as a second mark of a predetermined size to form the second label image.

[0075] In addition, the draft measurement system 200 may perform identification of the number area of ​​the draft mark and the corresponding number image when forming the first label image, and may perform calculation of the area between the numbers and the numbers corresponding to the intermediate values ​​existing between the numbers when forming the second label image.

[0076] Furthermore, when extracting a draft mark, the draft measurement system 200 may calculate the draft using a numerical ratio between a first label adjacent to the waterline and a second label adjacent to the first label based on a standard size of the draft mark. Alternatively, the draft mark may be extracted using a numerical ratio between a second label adjacent to the waterline and a first label adjacent to the second label based on a standard size of the draft mark.

[0077] Here, Mask R-CNN has a configuration in which a classification branch that predicts the class of an object, a bbox regression branch that performs bbox regression, and a mask branch that predicts a segmentation mask are added to the RoI (Region of Interest) obtained from the RPN of Faster R-CNN.

[0078] In addition, the mask branch is constructed by adding a small-sized fully convolutional network (FCN) to each RoI. To perform the segmentation task more effectively, a RoIAlign layer can be added to preserve the spatial location of objects.

[0079] Here, a segmentation mask refers to image segments obtained as a result of segmenting an image for each class, and the mask can effectively encode spatial information of objects in the image.

[0080] The RoIAlign layer can also be used to resolve misalignment between features and Regions of Interest (RoI).

[0081] This makes it possible to prevent image processing errors caused by factors such as the shooting angle, image quality, and the state of contamination on the hull surface, thereby improving the accuracy of draft measurement.

[0082] In the draft measurement system 200 of this embodiment, the draft measurement reference point definition unit 240 applies an artificial intelligence model including U-Net to detect the waterline, performs semantic segmentation to identify the position of the edge of the waterline, and defines the reference point for draft measurement by the intersection of a straight line selected by applying a voting algorithm based on the object detected by the object identification unit 220 and the segmented waterline with the segmented waterline.

[0083] That is, an artificial intelligence (AI) model including Mask R-CNN is applied to detect and identify the position of objects (draft marks) from each image extracted from the video. In addition, an artificial intelligence (AI) model including U-Net is applied to semantic segmentation for identifying the position of the edge of the waterline. In addition, a reference draft point is required to calculate the draft based on the image pixels from the object (draft mark) detected by the artificial intelligence (AI) recognition technology and the segmented waterline. Therefore, a voting algorithm is applied to specify the reference draft point along the line where the corresponding draft mark is located, and the intersection point between the straight line selected by the voting algorithm and the segmented waterline at the center lower part of the bounding box located directly above the waterline and surrounding the n marks recognized as objects is defined as the reference point for the draft calculation.

[0084] Moreover, in the draft measurement system 200 of this embodiment, the distortion correction unit 250 applies a scaling algorithm to correct distortion of the size of the draft mark caused by the photographing position.

[0085] Here, the term "scaling algorithm" refers to an algorithm that estimates the height of a mark based on the proportional change in the size of the mark across the entire frame of an image converted in frame units, and corrects distortion.

[0086] More specifically, the scaling algorithm is configured to model the proportional change in the size (in pixels) of the mark across frames of the frame-by-frame converted image as a function of the Y-axis (height) direction, and based thereon estimate the height (in pixels) of the mark along the Y-axis.

[0087] In addition, in this embodiment, the distortion correction unit 250 performs a perspective transform in which the surface of the ship with draft marks is regarded as a plane, and corrects the distortion in the X-axis direction by measuring the height of the waterline from the image corrected by the perspective transform, and also corrects the distortion in the Y-axis direction by predicting the value in the Y-axis direction based on the distortion correction in the X-axis direction.

[0088] In addition, in this embodiment, in order to monitor the value of the draft calculated by the calculation unit 270, a monitoring server 300 is provided that displays or stores the calculated value of the draft in real time.

[0089] That is, the monitoring server 300 includes a storage unit 310 that stores the calculated draft value so that the calculated draft value can be monitored in real time, and a display unit 320 that displays the calculated draft value.

[0090] Also, referring to FIG. 10, the draft measurement method of this embodiment includes an image capturing step S200 in which the image capturing device 110 captures an image of a draft mark including a number marked on the outside of the side of the ship and a water line surface, a preprocessing step S210 in which the image captured by the image capturing device 110 in the image capturing step S200 is received and the received image is converted into an image in reference frame units by a preprocessing unit 210 of the draft measurement system 200, an object identification step S220 in which the draft measurement system 200 detects the draft mark as an object from the image converted into the reference frame units in the preprocessing step S210 and specifies (identifies) its position, and a draft measurement system 200 detects the waterline from the image converted into the reference frame units in the preprocessing step S210. the draft measurement reference point definition step S230 for defining a reference point for draft measurement; a distortion correction step S240 for correcting distortion of the draft mark recognized in the object recognition step S220 and distortion of the reference point defined in the draft measurement reference point definition step S230; a measurement step S250 for measuring, by the draft measurement system 200, a distance from the draft mark corrected in the distortion correction step S240 to the draft measurement reference point corrected in the distortion correction step S240; an outlier removal step S260 for removing outliers from the measurement data measured in the measurement step S250 by the draft measurement system 200; and a calculation step S270 for calculating the value of the draft based on the data from which the outliers have been removed in the outlier removal step S260 by the draft measurement system 200.

[0091] In addition, the object recognition step S220 of the draft measurement method of this embodiment by the draft measurement system 200 of this embodiment uses the Mask R-CNN model to detect draft marks as objects from the image converted into reference frame units and identify their positions.

[0092] Further, the draft measurement reference point definition step S230 of the draft measurement method of this embodiment by the draft measurement system 200 of this embodiment includes a segmentation step S231 of performing semantic segmentation to identify the position of the edge of the waterline by applying an artificial intelligence model including U-Net in order to detect the waterline by the draft measurement system 200. In this case, the reference point for draft measurement is defined by the intersection point between the segmented waterline and a straight line selected by applying a voting algorithm based on the object detected in the object identification step S220 and the waterline segmented in the segmentation step S231.

[0093] Referring also to Fig. 12, the reference point of the draft measurement is used to estimate the position of the draft point along a straight line. To estimate the position of the draft point, the reference point of the draft measurement is located directly above the waterline, and the intersection point between the straight line selected by the voting algorithm and the segmented waterline is defined as the reference point for the draft calculation at the central lower part of the bounding box surrounding the n marks recognized as objects.

[0094] Here, the voting algorithm adopts a concept similar to majority voting. In the graph of Fig. 12(a), there are many lines connecting two points, and among them, there is a line that receives the most votes, and such a line is selected as the estimation line.

[0095] Therefore, even if the position of the draft mark in the image converted into the reference frame unit changes depending on the shooting position of the image, the draft measurement system 200 can use a voting algorithm to accurately define the draft point that serves as the reference for calculating the draft.

[0096] FIG. 13 shows the result of applying a voting algorithm to the draft measurement method of this embodiment.

[0097] Fig. 13(a) shows the state where the draft mark has been identified in the object identification step, and Fig. 13(b) shows the state where the last five characters have been selected by the voting algorithm.

[0098] 13(c) and 13(d) are diagrams showing the results of the draft marks and the reference draft points to which the voting algorithm has been applied.

[0099] In addition, in the distortion correction step S240 of the draft calculation method of this embodiment, a scaling algorithm is applied to correct distortion of the size of the draft mark due to the shooting position. The scaling algorithm estimates the height of the mark based on the proportional change in the size of the mark across the frames of the converted image on a frame-by-frame basis to correct the distortion.

[0100] Here, FIG. 14 is a diagram showing a reference for making draft marks on a flat surface, and FIG. 15 is a diagram showing a reference for making draft marks on a curved surface.

[0101] Also, FIG. 16(a) illustrates the change in character size due to perspective distortion, and FIG. 16(b) illustrates modeling to estimate character height (in pixels) as a function of the image y-axis value.

[0102] Therefore, the scaling algorithm is applied to calculate the draft by correcting distortions such as changes in the size of the draft mark due to the installation position depending on whether the draft mark is installed on a curved surface of the ship or on a flat surface, and changes in the size of the draft mark identified between image frames due to the user's shooting position during the draft calculation process by the draft measurement system.

[0103] That is, the scaling algorithm is configured to model the proportional change in the size (in pixels) of the mark across frames of the frame-by-frame converted image as a function of the Y-axis (height) direction, and based on that, estimate the height (in pixels) of the mark in any Y-axis direction.

[0104] Also, referring to FIG. 17, the outlier removal step S260 of the draft calculation method of this embodiment removes outliers from the results of analyzing each image frame to calculate the final draft value based on statistical univariate data such as standardized scores and interquartile ranges, and derives the final draft value as an arithmetic mean.

[0105] That is, the outlier removal step S260 applies an outlier criterion to the results of analyzing each image frame to calculate the final draft value, and removes outliers from the analysis results, thereby deriving a reliable final draft value as an arithmetic mean.

[0106] Furthermore, as a criterion for detecting abnormal values ​​in the primary analysis values, abnormal values ​​can be detected based on statistical univariate data such as standard scores and interquartile ranges.

[0107] Moreover, the draft calculation method of this embodiment further includes a monitoring step of transmitting the draft value calculated in the calculation step S270 to the monitoring server 300 for monitoring, and displaying or storing the draft value in real time.

[0108] According to the draft calculation method of this embodiment, the waterline can be measured more accurately and quickly than by visual measurement.

[0109] As described above, according to the draft measurement system and draft measurement method of this embodiment, artificial intelligence technology is applied to images extracted from video data, preventing image processing errors that occur due to the shooting angle, image quality, contamination state of the hull surface, etc., thereby improving the accuracy of draft measurement.

[0110] In addition, according to the draft measurement system and draft measurement method of this embodiment, the draft value can be confirmed based on video data captured in real time using a deep learning model for detecting the edges of draft marks and performing the segmentation required to identify the waterline surface, an edge detection algorithm and a distortion correction algorithm for detecting the waterline surface, as well as a drone and a small video camera, video reception / transmission and streaming technology.

[0111] In addition, the draft measurement system and draft measurement method of this embodiment can prevent accidents such as contusion, drowning, and musculoskeletal trauma that may occur when measuring draft, and can obtain more reliable and accurate measurement values ​​than visual measurement.

[0112] The above-described embodiments may be executed by various computer components, and may be implemented in the form of program instructions and recorded on a computer-readable recording medium. The computer-readable recording medium may store program instructions, data files, data structures, and the like, either alone or in combination. The program instructions recorded on the computer-readable recording medium may be specially designed and constructed for the present invention, or may be known to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic recording media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and special hardware devices such as ROMs, RAMs, and flash memories that are constructed to store and execute program instructions. Program instruction languages ​​include not only machine language codes created by compilers, but also high-level language codes that can be executed on a computer using an interpreter, etc. The hardware devices may be modified into one or more software modules to execute the processes according to the present invention, and vice versa.

[0113] The above-described embodiments are provided to allow those skilled in the art to easily understand the technical concept of the present invention, and should not be interpreted as limiting the present invention. It is obvious to those skilled in the art that the embodiments of the present invention can be modified and changed in various ways without departing from the concept and scope of the present invention. Therefore, such modifications and changes should be included in the scope of the claims of the present invention. [Explanation of symbols]

[0114] 50: Network 100: Drone 110: Camcorders and other video recording equipment 200: Waterline measurement system 210: Pre-processing section 220: Draft mark identification unit, object identification unit 230: Extraction unit, detection unit 240: Waterline detection section, draft measurement reference point definition section 250: Measurement section, distortion correction section 260: Distortion correction section, measurement section 270: Calculation section 300: Monitoring Server 310: Storage Department 320: Display section

Claims

1. At least one of a drone and a video camera that photographs the draft marks including the numbers marked on the outer side of the vessel and the waterline surface; and a draft measurement system that receives an image captured by at least one of the drone and the image capture device and calculates a draft; The draft measurement system comprises: A pre-processing unit converts the received video into an image in units of reference frames; and a draft mark identification unit that identifies a draft mark from the image converted into a reference frame unit by the preprocessing unit; and a waterline detection unit that detects a waterline from the image converted into reference frame units by the preprocessing unit; and A measurement unit that measures a pixel distance between the draft mark identified by the draft mark identification unit and the waterline detected by the waterline detection unit; and a distortion correction unit that corrects distortion in an X-axis direction and a Y-axis direction of the measurement data measured by the measurement unit; and A calculation unit that calculates a draft based on the correction data value corrected by the distortion correction unit. Draft measurement system based on image processing with artificial intelligence technology.

2. The waterline detection unit is An edge detection unit that detects an edge of a waterline from the image converted into the reference frame unit by the preprocessing unit, The waterline surface is identified by segmentation based on the edge of the waterline detected by the edge detection unit. A draft measurement system based on image processing to which the artificial intelligence technology according to claim 1 is applied.

3. The measurement unit includes: A first measuring unit that measures a pixel distance between the draft mark identified by the draft mark identifying unit and a draft mark above the waterline; and A second measurement unit that measures a pixel distance between the draft mark identified by the draft mark identification unit and a waterline, A draft measurement system based on image processing to which the artificial intelligence technology according to claim 1 is applied.

4. The distortion correction unit is A perspective transformation is performed to consider the surface of the ship with the draft marks as a plane, From the image corrected by this perspective transformation, the waterline height is measured and the distortion in the X-axis direction is corrected. A value in the Y-axis direction is predicted based on the correction of the distortion in the X-axis direction, and the distortion in the Y-axis direction is corrected. A draft measurement system based on image processing to which the artificial intelligence technology according to claim 1 is applied.

5. The present invention is characterized in that a monitoring server is further provided for at least one of displaying and storing the draft value in real time in order to monitor the draft value calculated by the calculation unit. A draft measurement system based on image processing to which the artificial intelligence technology according to claim 1 is applied.

6. At least one of a drone and a video camera that photographs the draft marks including the numbers marked on the outer side of the vessel and the waterline surface; and a draft measurement system that receives an image captured by at least one of the drone and the image capture device and calculates a draft; The draft measurement system comprises: A pre-processing unit converts the received video into an image in units of reference frames; and an object identification unit that detects draft marks as objects from the image converted into reference frame units by the preprocessing unit and identifies their positions; and a draft measurement reference point definition unit that detects a waterline from the image converted into reference frame units by the preprocessing unit and defines a reference point for draft measurement; and a distortion correction unit that corrects distortion of the draft mark identified by the object identification unit and distortion of the draft measurement reference point defined by the draft measurement reference point definition unit; and a measurement unit for measuring a distance between a draft mark whose distortion has been corrected by the distortion correction unit and a draft measurement reference point whose distortion has been corrected by the distortion correction unit; and A calculation unit that calculates a draft based on the measurement data value measured by the measurement unit. Draft measurement system based on image processing with artificial intelligence technology.

7. An image capturing step of capturing an image of a draft mark including a number marked on the outer side of the vessel and a waterline surface by at least one of a drone and an image capturing device; a pre-processing step of receiving the image captured in the image capturing step with a draft measurement system and converting the received image into an image in units of reference frames; a draft mark identification step of identifying draft marks from the image converted into a reference frame unit in the pre-processing step; and a waterline detection step of detecting a waterline from the image converted into a reference frame unit in the preprocessing step; and a measuring step of measuring a pixel distance between the draft mark identified in the draft mark identifying step and the waterline detected in the waterline detecting step; and a distortion correcting step of correcting distortion in the X-axis direction and distortion in the Y-axis direction of the measurement data measured in the measuring step; and A calculation step of calculating a draft based on the correction data corrected in the distortion correction step. Draft measurement method based on image processing using artificial intelligence technology.

8. The waterline detection step includes: The method includes an edge detection step of detecting an edge of a waterline in order to detect the waterline, and a step of identifying a waterline surface by segmentation based on the edge of the detected waterline. A draft measurement method based on image processing using artificial intelligence techniques as claimed in claim 7.

9. The measuring step includes: a first measuring step of measuring a pixel distance between the draft mark identified in the draft mark identifying step and a draft mark above the waterline; and a second measuring step of measuring a pixel distance between the draft mark identified in the draft mark identifying step and a waterline; A draft measurement method based on image processing using artificial intelligence techniques as claimed in claim 7.

10. The distortion correction step includes: a perspective transformation step in which the surface of the vessel with the draft marks is considered as a plane; and measuring the waterline height from the image corrected in the perspective transformation step, and correcting distortion in the X-axis direction; and correcting the distortion in the Y-axis direction by predicting a value in the Y-axis direction based on the correction of the distortion in the X-axis direction in the step of correcting the distortion in the X-axis direction. A draft measurement method based on image processing using artificial intelligence techniques as claimed in claim 7.

11. The method further includes a monitoring step of transmitting the draft value calculated in the calculation step to a monitoring server and at least one of displaying and storing the draft value in real time. A draft measurement method based on image processing using artificial intelligence techniques as claimed in claim 7.

12. An image capturing step of capturing an image of a draft mark including a number marked on the outer side of the vessel and a waterline surface by an image capturing device; a pre-processing step of receiving the image captured in the image capturing step with a draft measurement system and converting the received image into an image in units of a reference frame; an object identification step of detecting a draft mark as an object from the image converted into a reference frame unit in the preprocessing step and identifying its position; A draft measurement reference point definition step of detecting a waterline from the image converted into reference frame units in the preprocessing step and defining a reference point for draft measurement; a distortion correction step of correcting distortion of the draft marks identified in the object identification step and distortion of the draft measurement control points defined in the draft measurement control point definition step; and a measuring step of measuring a distance between a draft mark whose distortion has been corrected in the distortion correction step and a draft measurement reference point whose distortion has been corrected in the distortion correction step; and an outlier removal step of removing outliers from the data values ​​measured in the measurement step; and A calculation step of calculating a draft based on the data from which the abnormal values ​​have been removed in the abnormal value removal step. Draft measurement method based on image processing using artificial intelligence technology.

13. The object identification step includes: The method is characterized in that a Mask R-CNN model is applied to detect draft marks as objects from images converted into reference frame units and identify their positions. A method for draft measurement based on image processing applying artificial intelligence techniques according to claim 12.

14. The draft measurement reference point definition step includes: To detect the waterline, an artificial intelligence model including U-Net is applied to perform semantic segmentation to locate the waterline edge; A draft measurement reference point is defined by an intersection point between a straight line selected by applying a voting algorithm based on the object detected in the object identification step and the semantic segmented waterline and the semantic segmented waterline. A method for draft measurement based on image processing applying artificial intelligence techniques according to claim 12.

15. The distortion correction step includes: A scaling algorithm is applied to correct the distortion of the draft mark size due to the shooting position, the scaling algorithm estimates the height of the draft marks based on the proportional change in size of the draft marks across the entire image in each frame to correct for distortion; A method for draft measurement based on image processing applying artificial intelligence techniques according to claim 12.

16. The outlier removal step includes: The method is characterized in that outliers are detected and removed from the results of analyzing each image frame by frame based on statistical univariate data of standard scores or interquartile ranges, and the draft value is derived as an arithmetic mean. A method for draft measurement based on image processing applying artificial intelligence techniques according to claim 12.

17. The method further includes a monitoring step of transmitting the draft value calculated in the calculation step to a monitoring server and at least one of displaying and storing the draft value in real time. A method for draft measurement based on image processing applying artificial intelligence techniques according to claim 12.

18. A computer-readable recording medium having recorded thereon a computer program for causing a computer to execute a draft measurement method based on image processing to which the artificial intelligence technology according to any one of claims 7 to 17 is applied.

Images