Draft measuring program and draft measuring device

Abstract

A draft measuring program according to the present application causes a computer to execute: an acquisition procedure for acquiring a hull image obtained by photographing a hull; a region detection procedure for detecting a water surface region in the hull image using a detector that takes the hull image as an input; a reading procedure for reading scale numerical values indicated by draft marks in the hull image using a recognizer that takes the hull image as an input; and a measurement procedure for measuring the draft of the hull on the basis of the detection result of the water surface region and the reading result of the scale numerical values.

Description

Draft measurement program and draft measurement device

[0001] The present invention relates to a draft measurement program and a draft measurement device.

[0002] Conventionally, methods for detecting the waterline of a ship have been proposed. For example, Patent Document 1 proposes a waterline detection method that enables detection of the waterline not only when the ship is stationary but also when the ship is moving and detecting the waterline from bow to stern.

[0003] Patent No. 6948554

[0004] However, the above-mentioned conventional techniques do not always realize easy and highly accurate draft measurement. For example, the above-mentioned conventional techniques provide a temperature measurement range on the inner surface of the outer plating of the hull within a predetermined range in the vertical direction from the expected draft surface, measure the temperature within the temperature measurement range, and detect the draft surface from the position where the measured temperature changes and / or the temperature distribution.

[0005] In such conventional technology, a system for detecting the waterline level must be installed in the hull in advance. For example, heaters must be attached at a predetermined distance on both sides of the temperature measurement range to heat the periphery or both sides of the temperature measurement range on the inner surface of the ship's shell. This is thought to require a variety of work loads before the waterline level can actually be detected. Therefore, there is a need for an easy and highly accurate draft measurement method.

[0006] Therefore, in order to solve the above problems and achieve the object, the draft measurement program of the present invention is characterized in that it causes a computer to execute an acquisition procedure for acquiring a hull image in which the hull has been photographed, an area detection procedure for detecting the water surface area in the hull image using a detector that receives the hull image as input, a reading procedure for reading the scale numbers indicated by the draft marks in the hull image using a recognizer that receives the hull image as input, and a measurement procedure for measuring the draft of the hull based on the detection result of the water surface area and the reading result of the scale numbers.

[0007] According to the present invention, draft measurement can be easily performed with high accuracy.

[0008] FIG. 1 is a diagram showing an example of a system according to an embodiment. FIG. 2 is a diagram showing an overall image of draft measurement processing according to an embodiment. FIG. 3 is a diagram showing an example of the configuration of a draft measurement device according to an embodiment. FIG. 4 is a diagram showing an example of the configuration of a learning device according to an embodiment. FIG. 5 is a flowchart showing an overall processing procedure of draft measurement processing according to an embodiment. FIG. 6 is a flowchart showing a mark detection processing procedure according to an embodiment. FIG. 7 is a flowchart showing a water surface area detection processing procedure according to an embodiment. FIG. 8 is a flowchart showing a scale numerical value reading processing procedure according to an embodiment. FIG. 9 is a flowchart showing a draft measurement processing procedure according to an embodiment. FIG. 10 is a diagram showing an example of the configuration of a draft measurement device according to a modified embodiment. FIG. 11 is a block diagram showing an example of the hardware configuration of a computer corresponding to the draft measurement device according to an embodiment.

[0009] Hereinafter, an embodiment of the present invention will be described in detail with reference to the accompanying drawings. Note that the draft measurement program and draft measurement device according to the present disclosure are not limited to the embodiment. In the following embodiments, the same components are designated by the same reference numerals, and redundant description will be omitted.

[0010] Furthermore, one or more embodiments (including examples, modifications, and application examples) described below can be implemented independently. However, at least a portion of the embodiments described below may be implemented in appropriate combination with at least a portion of another embodiment. These embodiments may include novel features that are different from one another. Therefore, these embodiments may contribute to solving different objectives or problems and may achieve different effects.

[0011] (Embodiment) [1. Introduction] In conventional draft measurement, workers repeatedly visually read draft marks (scales that indicate the sinking state of the hull) marked on the hull, which has led to problems with poor work efficiency. Furthermore, draft marks are often marked in multiple locations per ship (e.g., the bow, the center of the hull, the stern, etc.), which further increases the workload of workers involved in draft measurement.

[0012] Furthermore, visually reading draft marks requires skilled work, and reading accuracy is poor in bad weather (e.g., rough waves), so there is a demand for highly accurate draft measurements that can be performed regardless of the worker's level of skill or the weather.

[0013] Therefore, the inventors of the present invention focused on the fact that by applying image processing using AI (Artificial Intelligence) to draft measurement, workers would be able to measure the draft easily and with high accuracy simply by photographing the hull.

[0014] Specifically, the technology proposed in this invention is implemented as a program (application) installed on an information processing terminal (mobile terminal) used by workers. This allows the information processing terminal to input a photographed hull image into a detector to detect the water surface area in the hull image, and to input the hull image into a recognizer to read the scale values ​​of the draft marks marked on the hull. Furthermore, the information processing terminal can measure the hull draft based on the water surface area detection result and the scale value reading result.

[0015] For this reason, by using an information processing terminal equipped with the draft measurement program of the present invention, workers can obtain draft measurement results semi-automatically, for example, by simply taking a hull image of a ship stopped at a port or the like, including the draft marks marked on the hull.

[0016] Furthermore, as will be described later, the draft measurement program of the present invention performs draft measurements for each of the hull images taken over a predetermined period of time in chronological order, and calculates the final draft to be presented to the worker based on the draft measurement results for each hull image. This process solves the problem of unstable drafts caused by the hull swaying due to waves, thereby reducing the workload on the workers and further improving the accuracy of draft measurements.

[0017] The use of the draft measurement program according to the present invention is not limited to ships parked in ports, etc. For example, the draft measurement program according to the present invention can also be used for ships parked outside ports or ships drifting (drifting in the waters near a port in order to adjust the arrival time at the port). In this way, the use of the draft measurement program according to the present invention is not limited to a specific scenario, and can be used in any scenario as long as the target ship is parked in a stationary state.

[0018] 2. System Configuration Next, the configuration of a system according to an embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram showing an example of a system according to an embodiment. Fig. 1 shows a system 1 as an example of a system according to an embodiment. Draft measurement processing according to an embodiment is realized in the system 1.

[0019] 1 , the system 1 includes a learning device 50 and a draft measurement device 100. The learning device 50 and the draft measurement device 100 are connected to each other via a network N so that they can communicate with each other via a wired or wireless connection. The system 1 may include any number of learning devices 50 and any number of draft measurement devices 100.

[0020] The learning device 50 learns a "water surface area detection model MD1" as a detector used to detect water surface areas. For example, the learning device 50 uses, as learning data (correct answer data), photographed images in which a hull is photographed to include a water surface area and in which the water surface area is defined, to generate a water surface area detection model MD1 that has learned the relationship between the state of the hull at the time of photographing and the state of the water surface area. For example, the learning device 50 generates the water surface area detection model MD1 by having a model learn the relationship between the pattern of the hull and the pattern of the water surface area based on hull images photographed in various situations. For example, the learning device 50 may generate the water surface area detection model MD1 using a learning technique such as deep learning. According to the water surface area detection model MD1, a hull image photographed at this time is input, and a hull image in which the water surface area has been determined is output.

[0021] The learning device 50 may also generate an "OCR (Optical Character Reader) model MD2," which is a character recognizer used to read the scale indicated by the draft marks. For example, the learning device 50 can improve character recognition accuracy by combining conventional OCR with machine learning, a feature of AI. Note that the learning device 50 does not necessarily have to generate the OCR model MD2. In this case, the system 1 according to the embodiment uses an externally generated OCR model MD2.

[0022] The draft measurement device 100 is a central device that performs draft measurement processing according to the embodiment. Specifically, the draft measurement device 100 detects the water surface area in the hull image using a water surface area detection model MD1, which receives a photographed hull image as input. The draft measurement device 100 also reads the scale values ​​indicated by the draft marks in the hull image using an OCR model MD2, which receives the hull image as input. The draft measurement device 100 then measures the hull draft based on the water surface area detection result and the scale value reading result.

[0023] For example, the draft measurement device 100 may be an information processing terminal used by a worker. Examples of information processing terminals include smartphones, wearable devices, tablet terminals, laptop personal computers (PCs), desktop PCs, mobile phones, and personal digital assistants (PDAs). It is expected that workers will use the information processing terminal to take images of the ship from a pier near where the ship is moored or from the sea after using a small boat to go around the side of the ship. Therefore, a smartphone, which is easy to use, is particularly preferable. The location where the worker takes the images is not limited to a pier or the sea. For example, the worker may take the images from a quay near where the ship is moored, or may take the images using a draft check ladder or drone.

[0024] Furthermore, the draft measurement device 100 is capable of executing the above-described draft measurement process by installing an application AP (abbreviated as "app AP") as a draft measurement program according to the embodiment. On the other hand, the draft measurement device 100 may be newly manufactured as a device dedicated to performing the draft measurement process according to the embodiment. In this embodiment, the draft measurement device 100 will be described as having the app AP installed.

[0025] 3. Overview of Draft Measurement Processing Next, an overview of the draft measurement processing implemented by the app AP will be described. FIG. 2 is a diagram showing an overview of the draft measurement processing according to the embodiment. FIG. 2 shows a scene in which an operator OP performs work to measure the draft DR of a hull SH using a draft measurement device 100 in which the app AP is installed. For example, the operator OP uses an imaging function activated by the app AP to capture an image of the hull SH for a predetermined period (e.g., a period of about 30 seconds to 1 minute) so as to include the draft marks MK and the water surface area AR (e.g., the boundary between the hull SH and the sea).

[0026] In this case, the draft measurement device 100 sequentially acquires hull images SG captured continuously in real time using its imaging function in accordance with the chronological order of the real-time capture. Each time a hull image SG is acquired, the draft measurement device 100 performs draft measurement processing for each acquired hull image SG, and when individual draft measurement results for each hull image SG for a predetermined period have been obtained, these results are integrated to calculate the final draft DR value. An example of the draft measurement processing will be described below, focusing on one of the most recent acquired hull images SG.

[0027] First, the hull image SG may contain multiple draft marks MK. Therefore, when the draft measurement device 100 acquires the hull image SG, it extracts an image of a range focusing only on the draft marks MK from the acquired hull image SG. Specifically, the draft measurement device 100 cuts out (crops) the image focusing on the draft marks MK from the hull image SG.

[0028] The draft measurement device 100 inputs an extracted image EXG extracted from a hull image SG into a water surface area detection model MD1 to obtain a detected extracted image EXG in which the water surface area AR is detected. Next, the draft measurement device 100 inputs the detected extracted image EXG into an OCR model MD2 to obtain a read extracted image EXG in which the scale numerical values ​​NM of each draft mark MK included in the extracted image EXG have been read.

[0029] Here, the draft measurement device 100 detects the water surface area AR through a two-stage inference process using a water surface area detection model MD1 and an OCR model MD2, and can output a processed extraction image EXGn as an extraction image EXG in which the scale numerical value NM of the draft mark MK is read.

[0030] As a result, the draft measurement device 100 can identify the waterline WL from the processed extracted image EXGn, as shown in Fig. 2. Therefore, the draft measurement device 100 measures the draft DR based on the scale value NM determined from the positional relationship between the waterline WL and the draft mark MK in the processed extracted image EXGn. Fig. 2 shows an example in which the draft measurement device 100 calculates "10m 50cm" as the measurement result of the draft DR.

[0031] The draft DR is calculated for each hull image SG acquired sequentially in accordance with the time series of real-time photography. Therefore, for example, after the draft measurement device 100 has finished measuring the draft DR individually for all hull images SG for a predetermined period, it calculates a moving average DRav of the measurement results RE of the draft DR based on the time series data TS of the measurement results RE. The draft measurement device 100 then acquires the moving average DRav as the final measurement result REx to be presented to the operator OP. As a result, the draft measurement device 100 displays the final measurement result REx, for example, on a display screen of the device itself.

[0032] 4. Configuration of the Draft Measurement Device The draft measurement device 100 according to the embodiment will be described with reference to Fig. 3. Fig. 3 is a diagram showing an example of the configuration of the draft measurement device 100 according to the embodiment. As shown in Fig. 3, the draft measurement device 100 has a communication unit 110, a storage unit 120, and a control unit 130.

[0033] (Regarding the communication unit 110) The communication unit 110 is realized by, for example, a network interface card (NIC), etc. For example, the communication unit 110 is connected to the network N via a wired or wireless connection, and transmits and receives information to and from the learning device 50, for example.

[0034] (Regarding the storage unit 120) The storage unit 120 is realized by, for example, a semiconductor memory element such as a random access memory (RAM) or a flash memory, or a storage device such as a hard disk or an optical disk. The storage unit 120 may store, for example, data related to the information processing according to the embodiment and a draft measurement program. According to the example of FIG. 3, the storage unit 120 includes a model data storage unit 121, an image data storage unit 122, and a result data storage unit 123.

[0035] The model data storage unit 121 stores a water surface area detection model MD1 and an OCR model MD2.

[0036] The image data storage unit 122 stores hull images SG obtained by photographing the hull SH. For example, the image data storage unit 122 stores the time at which the hull SH was photographed in association with the hull image SG acquired at that time. The image data storage unit 122 may also associate an extracted image EXG extracted from the hull image SG with a combination of the time and the hull image SG.

[0037] The result data storage unit 123 stores information about the water surface area AR detected using the water surface area detection model MD1 (detection result of the water surface area AR) and information about the scale values ​​NM of the draft marks MK read using the OCR model MD2 (read result of the scale values ​​NM). For example, the result data storage unit 123 may store this information in chronological order for each hull image SG. The result data storage unit 123 may also store measurement results RE in which the draft DR is measured individually for each hull image SG, and final measurement results REx calculated as a moving average DRav of the measurement results RE.

[0038] (Regarding the control unit 130) The control unit 130 is realized by a CPU (Central Processing Unit), an MPU (Micro Processing Unit), or the like using RAM as a work area to execute various programs (for example, the draft measurement program according to the embodiment) stored in a storage device inside the draft measurement device 100. The control unit 130 is also realized by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

[0039] As shown in Fig. 3, the control unit 130 has an acquisition unit 131, a mark detection unit 132, an area detection unit 133, a reading unit 134, an error control unit 135, a measurement unit 136, and a presentation unit 137, and realizes or executes the functions and actions of the information processing described below. Note that the internal configuration of the control unit 130 is not limited to the configuration shown in Fig. 3, and may be other configurations as long as they perform the information processing described below. Furthermore, the connection relationships between the processing units of the control unit 130 are not limited to the connection relationships shown in Fig. 3, and may be other connection relationships.

[0040] (Acquisition unit 131) The acquisition unit 131 acquires a hull image SG obtained by capturing the hull SH of the ship whose draft is being measured. The hull image SG corresponds to one frame image among a group of images captured in real time of the hull SH in chronological order, i.e., among time-series images. Therefore, the acquisition unit 131 sequentially acquires frame images according to the time series of real-time capture.

[0041] (Mark detection unit 132) The mark detection unit 132 detects draft marks MK included in the hull image SG using a draft mark detection model MD3 that receives the hull image SG as input. For example, the mark detection unit 132 acquires a detection result in which the draft marks MK are detected by the draft mark detection model MD3. The draft mark detection model MD3 can detect a predetermined range in the hull image SG that includes the draft marks MK using any image analysis technology.

[0042] (Area detection unit 133) The area detection unit 133 detects the water surface area AR in the input image using the water surface area detection model MD1. More specifically, the area detection unit 133 inputs an extraction image EXG, in which a predetermined range including the draft mark MK is extracted from the hull image SG, into the water surface area detection model MD1, to obtain a detected extraction image EXG in which the water surface area AR has been detected.

[0043] In addition, in the case where the hull SH is photographed in real time in a chronological order, the area detection unit 133 detects the water surface area AR in each extracted image EXG extracted from each hull image SG corresponding to one frame image.

[0044] (Reading unit 134) The reading unit 134 reads the scale numbers NM indicated by the draft marks MK in the input image using the OCR model MD2. More specifically, the reading unit 134 inputs the detected extraction image EXG in which the water surface area AR has been detected into the OCR model MD2, thereby acquiring the read extraction image EXG in which the scale numbers NM of the draft marks MK included in the extraction image EXG have been read.

[0045] Furthermore, in cases where the hull SH is photographed in real time in a chronological order, the reading unit 134 reads the scale values ​​NM for each extracted image EXG extracted from each hull image SG corresponding to one frame image. If there are any read results for which the reliability does not satisfy a threshold value among the read results of the scale values ​​NM, the reading unit 134 may correct the read results for which the reliability does not satisfy the threshold value based on the read results for which the reliability does satisfy the threshold value. The reliability is calculated for each draft mark MK during character recognition processing using the OCR model MD2, for example.

[0046] (Error control unit 135) If the area detection unit 133 fails to detect the water surface area AR or if the reading unit 134 fails to read the scale value NM, the error control unit 135 estimates the cause of the failure and causes the operator OP to present solutions to the estimated cause. It is assumed that it is statistically clear that detection of the water surface area AR or reading of the scale value NM will fail if the shooting position is too far from the hull SH or if the captured image is unclear due to camera shake. In such cases, the error control unit 135 can present a message such as, "You are too far from the subject, so please move closer and take the image," or "The image is unclear, so please take the image in a stable state."

[0047] (Measurement Unit 136) The measurement unit 136 measures the draft DR of the hull SH based on the detection results of the water surface area AR and the reading results of the scale values ​​NM. For example, the measurement unit 136 determines the coordinates on the extracted image EXG of the intersection between the waterline WL indicated by the detection results of the water surface area AR and a line tracing the draft marks MK, and measures the reading results of the scale values ​​NM indicated by the draft marks MK corresponding to these coordinates as the draft DR of the hull SH. Furthermore, the measurement unit 136 measures the draft DR of the hull SH for each extracted image EXG as the hull SH is photographed in real time in chronological order, and therefore calculates a moving average DRav of the measurement results RE based on the measurement results RE of the draft DR obtained for each extracted image EXG. Then, the measurement unit 136 acquires the moving average DRav as the final measurement result REx to be presented to the operator OP.

[0048] (Presentation Unit 137) The presentation unit 137 presents to the operator OP the value of the draft DR measured by the measurement unit 136. For example, the presentation unit 137 displays the final measurement result REx (moving average DRav) on a display screen.

[0049] 5. Configuration of the Learning Device A learning device 50 according to an embodiment will be described with reference to Fig. 4. Fig. 4 is a diagram showing an example configuration of the learning device 50 according to an embodiment. As shown in Fig. 4, the learning device 50 includes a communication unit 51, a storage unit 52, and a control unit 53.

[0050] (Regarding the communication unit 51) The communication unit 51 is realized by, for example, a NIC etc. For example, the communication unit 51 is connected to the network N by wire or wirelessly, and transmits and receives information to and from the draft measurement device 100, for example.

[0051] (Regarding the storage unit 52) ​​The storage unit 52 is realized by, for example, a semiconductor memory element such as a RAM or a flash memory, or a storage device such as a hard disk or an optical disk. According to the example of Fig. 4, the storage unit 120 has a training data storage unit 52a and a model data storage unit 52b.

[0052] The learning data storage unit 52a stores learning data used to generate the water surface area detection model MD1. As described above, the learning data may be photographed images of various types of ship hulls SH photographed from various angles so as to include the water surface area AR, and the position (range) of the water surface area AR in the photographed image may be defined. The model data storage unit 52b stores the trained model.

[0053] (Regarding the control unit 53) The control unit 53 is realized by a CPU, an MPU, or the like, executing various programs stored in a storage device inside the learning device 50 using RAM as a work area. The control unit 53 is also realized by an integrated circuit such as an ASIC or FPGA, for example.

[0054] As shown in Fig. 4, the control unit 53 has an acquisition unit 53a, a learning unit 53b, and a transmission unit 53c, and realizes or executes the functions and actions of the information processing described below. Note that the internal configuration of the control unit 53 is not limited to the configuration shown in Fig. 4, and may be other configurations as long as they perform the information processing described below. Furthermore, the connection relationship between the processing units of the control unit 53 is not limited to the connection relationship shown in Fig. 4, and may be other connection relationships.

[0055] (Acquisition Unit 53a) The acquisition unit 53a acquires learning data. For example, the acquisition unit 53a acquires learning data from a submission source. The acquisition unit 53a also acquires learning data from an external device that stores the learning data.

[0056] (Learning unit 53b) The learning unit 53b generates the water surface area detection model MD1 by training the model on training data. Specifically, the learning unit 53b trains the model to understand the relationship between the state of the hull SH and the state of the water surface area AR at the time of image capture. For example, the learning unit 53b trains the model to understand the relationship between the pattern of the hull SH and the pattern of the water surface area AR based on hull images SG captured in various situations.

[0057] (Transmitter 53 c) The transmitter 53 c transmits the water surface area detection model MD1 to the draft measurement device 100.

[0058] 6. Operational Processing Next, the operational processing of the draft measurement device 100 that operates under the control of the application AP will be specifically described with reference to the drawings. In the following description, the expression "extracting an image" can be replaced with the expressions "cutting out an image" or "cropping an image."

[0059] 6-1. Overall Processing Procedure First, the overall processing procedure of the draft measurement process according to the embodiment will be described with reference to Fig. 5. Fig. 5 is a flowchart showing the overall processing procedure of the draft measurement process according to the embodiment.

[0060] The acquisition unit 131 acquires the earliest hull image SG1 in chronological order at the present time (step S501).

[0061] The mark detection unit 132 executes a draft mark detection process using the hull image SG1 acquired in step S501 (step S502). Specifically, the mark detection unit 132 executes a draft mark detection process to detect draft marks MK included in the hull image SG1. The detailed processing procedure of the mark detection process will be described with reference to FIG. 6.

[0062] The area detection unit 133 acquires an extracted image EXG1 in which a predetermined range including the draft marks MK is extracted from the hull image SG1 based on the detection results of the draft marks MK (step S503).

[0063] Then, the area detection unit 133 executes a water surface area detection process using the extracted image EXG1 (step S504). Specifically, the area detection unit 133 executes the water surface area detection process to detect the water surface area AR from the extracted image EXG1. The detailed processing procedure of the water surface area detection process will be described with reference to FIG. 7.

[0064] Here, the error control unit 135 determines whether or not the area detection unit 133 has succeeded in detecting the water surface area AR based on the acquisition status of the detection result of the water surface area AR (step S505).

[0065] If the area detection unit 133 successfully detects the water surface area AR (step S505; Yes), the reading unit 134 acquires an extracted image EXG12 in which the water surface portion is filled in based on the detection result of the water surface area AR (step S506).

[0066] Then, the reading unit 134 executes a scale value reading process using the extracted image EXG12 (step S507). Specifically, the reading unit 134 executes the scale value reading process to read the scale values ​​NM of the draft marks MK (draft marks MK detected in step S502) included in the extracted image EXG12. The detailed processing procedure of the scale value reading process will be described with reference to FIG. 8.

[0067] Furthermore, the error control unit 135 determines whether or not the reading unit 134 has successfully read the scale value NM based on the acquisition status of the detection result of the water surface area AR (step S508).

[0068] If the reading unit 134 has successfully read the scale numerical value NM (step S508; Yes), the measurement unit 136 executes a draft measurement process using the extraction image EXG13 with the reading result attached (step S509). Specifically, the measurement unit 136 measures the draft DR of the hull SH based on the detection result of the water surface area AR included in the extraction image EXG13 and the reading result of the scale numerical value NM included in the extraction image EXG13. The detailed processing procedure of the draft measurement process will be described with reference to FIG. 9.

[0069] The measurement unit 136 then determines whether or not the measurement of the draft DR for each of the hull images SG for the predetermined period has been completed (step S510). If the measurement of the draft DR for all of the hull images SG for the predetermined period has not been completed (step S510; No), the measurement unit 136 returns to step S501.

[0070] On the other hand, when the measurement of the draft DR for all hull images SG for the predetermined period has been completed (step S510; Yes), the measurement unit 136 calculates a moving average DRav of the measurement results RE based on the measurement results RE of the draft DR obtained for each extracted image EXG13 for the predetermined period (step S511). Note that the measurement unit 136 may exclude measurement results RE estimated to be outliers from the measurement results RE for each extracted image EXG13, and calculate the moving average DRav using the remaining measurement results RE.

[0071] Then, the measurement unit 136 acquires the moving average DRav as the final measurement result REx to be presented to the operator OP, and presents the measurement result REx to the operator OP (step S512).

[0072] In addition, if the area detection unit 133 fails to detect the water surface area AR (step S505; No) or if the reading unit 134 fails to read the scale value NM (step S508; No), the error control unit 135 estimates the cause of the failure and presents a message to the worker OP according to the estimated cause (step S513).

[0073] 6-2. Mark Detection Processing Procedure Next, the procedure of the mark detection processing performed by the mark detection unit 132 will be described with reference to Fig. 6. Fig. 6 is a flowchart showing the procedure of the mark detection processing according to the embodiment.

[0074] The mark detection unit 132 receives the hull image SG1 as input and calls the draft mark detection model MD3 (step S601).

[0075] The mark detection unit 132 then acquires the detection result of the draft marks MK included in the hull image SG1 detected by the draft mark detection model MD3 (step S602). Specifically, the mark detection unit 132 acquires the hull image SG1 in which the positions of the predetermined ranges MKAR including the draft marks MK are marked, as shown in FIG. 6(a).

[0076] Next, the mark detection unit 132 edits the size of the range MKAR and extracts images included in the edited range MKAR from the hull image SG1 (step S603). For example, the mark detection unit 132 may edit the size of the range MKAR so as to satisfy the following conditions:

[0077] (i) 416 x 416 or larger (ii) Aspect ratio 1:1 (iii) Both length and width are multiples of 16

[0078] The mark detection unit 132 transmits the extracted image EXG1, which is an image extracted from the hull image SG1 in accordance with the edited range MKAR, to the area detection unit 133 (step S604). As a result, as described in step S503 of FIG. 5, the area detection unit 133 acquires the extracted image EXG1.

[0079] 6-3. Water Surface Area Detection Processing Procedure The procedure of the water surface area detection processing performed by the area detection unit 133 will be described with reference to Fig. 7. Fig. 7 is a flowchart showing the water surface area detection processing procedure according to the embodiment.

[0080] When the area detection unit 133 acquires the extraction image EXG1, it extracts an image of the vicinity of the water surface that is focused closer to the water surface from the extraction image EXG1, as shown in FIG. 7(a) (step S701).

[0081] The area detection unit 133 receives an extracted image EXG11, which is an image of the water surface and its vicinity extracted from the extracted image EXG1, as an input, and calls a water surface area detection model MD1 (step S702).

[0082] Then, the area detection unit 133 acquires the detection result of the water surface area AR included in the extracted image EXG11 detected by the water surface area detection model MD1 (step S703). Specifically, the area detection unit 133 acquires the extracted image EXG11 in which the position of the water surface area AR is marked.

[0083] If the area detection unit 133 can acquire an extraction image EXG11 in which the position of the water surface area AR is marked, it may output information indicating successful detection to the error control unit 135. If the area detection unit 133 cannot acquire an extraction image EXG11 in which the position of the water surface area AR is marked, it may output information indicating unsuccessful detection to the error control unit 135. The error control unit 135 performs the determination process of step S505 based on this information.

[0084] Next, the area detection unit 133 generates contour information WO of the water surface portion indicated by the water surface area AR based on the detection result of the water surface area AR and the extracted image EXG11 (step S704).

[0085] The area detection unit 133 also generates an image WG1 of the water surface portion indicated by the water surface area AR based on the detection result of the water surface area AR and the extraction image EXG11 (step S705). For example, as shown in Figure 7(b), the area detection unit 133 generates the image WG1 of the water surface portion on the extraction image EXG11 by filling in with black the area on the extraction image EXG11 that is formed by the position coordinates indicated by the contour information WO.

[0086] The area detection unit 133 transmits the extracted image EXG12, in which the image WG1 of the water surface portion has been generated from the extracted image EXG11, to the reading unit 134 (step S706). As a result, as described in step S506 of FIG. 5, the reading unit 134 obtains the extracted image EXG12 in which the water surface portion is filled in.

[0087] 6-4. Scale Value Reading Process Procedure The procedure of the scale value reading process performed by the reading unit 134 will be described with reference to Fig. 8. Fig. 8 is a flowchart showing the scale value reading process procedure according to the embodiment.

[0088] When the reading unit 134 acquires the extraction image EXG12, it inputs the extraction image EXG12 and calls the OCR model MD2 (step S801).

[0089] The reading unit 134 obtains the reading result of the scale numerical values ​​NM of the draft marks MK included in the extraction image EXG12 read by the OCR model MD2 (step S802). Specifically, as shown in FIG. 8A, the reading unit 134 obtains the extraction image EXG12 in which the reading result of the scale numerical values ​​NM is written for each draft mark MK.

[0090] 8A shows an example in which, for each draft mark MK surrounded by a bounding box BX, the read result of the scale numerical value NM indicated by that draft mark MK is displayed on the extraction image EXG12. For example, when attention is focused on a draft mark MK showing "17M" as the scale numerical value NM, an example is shown in which "1700" is displayed in association with "17M" as the read result. Also, when attention is focused on a draft mark MK showing "8" immediately below "17M" as the scale numerical value NM, an example is shown in which "1680" is displayed in association with "8" as the read result.

[0091] For this reason, the reading unit 134 may convert the unit of the reading result from "m" to "cm" and generate an extraction image EXG13 in which the converted reading result is displayed for the extraction image EXG12.

[0092] According to the OCR model MD2, a reliability indicating the accuracy of the reading result of the scale numerical value NM is output for each draft mark MK during the character recognition process. Therefore, if there are reading results of the scale numerical value NM whose reliability does not satisfy the threshold, the reading unit 134 corrects the reading results whose reliability does not satisfy the threshold based on the reading results whose reliability satisfies the threshold (step S803).

[0093] For example, the draft mark MK may be affected by dirt, peeling paint, or other factors, resulting in missing characters or the addition of unnecessary lines. In such cases, the accuracy of character recognition by the OCR model MD2 may decrease, resulting in an output reliability that does not satisfy the threshold. Therefore, the reading unit 134 may correct a reading result whose reliability does not satisfy the threshold with a reading result whose reliability does satisfy the threshold.

[0094] For example, suppose that the reading result of the draft mark MK indicating "8" is "1680," the reading result of the draft mark MK indicating "6" is also "1680," and the reading result of the draft mark MK indicating "4" is "1640." Also, suppose that the reliability of the reading result "1680" of the draft mark MK indicating "6" does not satisfy the threshold. In such a case, the reading unit 134 can correct the reading result of the draft mark MK indicating "6" to "1660" based on the fact that the reading results of the draft mark MK indicating "8" and the reading results of the draft mark MK indicating "4" are reliable and that the scale numerical values ​​NM are given at intervals of "20 cm."

[0095] Furthermore, if there are two or more reading results of the scale numerical value NM (for example, reading results whose reliability satisfies a threshold), the reading unit 134 may output information indicating successful reading to the error control unit 135, and if there are less than two reading results of the scale numerical value NM, the reading unit 134 may output information indicating unsuccessful reading to the error control unit 135. The error control unit 135 performs the determination process of step S508 based on this information.

[0096] Furthermore, reading unit 134 transmits extraction image EXG13, which is extraction image EXG12 with the reading result attached, to measurement unit 136 (step S804). As a result, as described in step S509 of FIG. 5, measurement unit 136 acquires extraction image EXG13 with the reading result attached.

[0097] 6-5. Draft Measurement Processing Procedure The procedure of the draft measurement processing performed by the measurement unit 136 will be described with reference to Fig. 9. Fig. 9 is a flowchart showing the draft measurement processing procedure according to the embodiment.

[0098] When the measurement unit 136 acquires the extracted image EXG13, it calculates a straight line LN1 that passes through the lower right corner of the bounding box BX, as shown in Fig. 9A (step S901). For example, the measurement unit 136 calculates the straight line LN1 in the coordinate system of the extracted image EXG13. Note that the straight line LN1 can be said to be a straight line that traces the draft marks MK in a single line.

[0099] Next, the measurement unit 136 identifies the waterline WL1 based on the contour information WO included in the image WG1 of the water surface portion (step S902).

[0100] Then, the measurement unit 136 calculates the coordinates of the intersection P between the waterline WL1 and the straight line LN1 on the extracted image EXG13 (step S903).

[0101] The measurement unit 136 measures the draft DR based on the positional relationship between the coordinates of the intersection point P and the read results of the scale numerical values ​​NM (step S904). For example, the measurement unit 136 acquires, from the read results of the scale numerical values ​​NM, the read results that exist on the coordinates of the intersection point P as the measurement result RE of the draft DR. According to the example of Fig. 9(a), the measurement unit 136 can acquire "16m00cm" as the measurement result RE.

[0102] At this point, the measurement unit 136 has finished measuring the draft DR for one extracted image EXG13 obtained from one hull image SG1 (frame image). Therefore, the measurement unit 136 proceeds to step S510 in FIG. 5 (step S905).

[0103] 7. Modifications The draft measurement device 100 according to the embodiment may be implemented in various different forms other than the above-described embodiment. Therefore, other embodiments of the draft measurement device 100 according to the embodiment will be described below.

[0104] 7-1. Image Correction Fig. 10 shows an example of the configuration of a draft measurement device 100A according to a modified embodiment. The draft measurement device 100A is configured by adding an image correction unit 138 to the draft measurement device 100 shown in Fig. 3. The image correction unit 138, like the other processing units, is realized by executing the application AP within the draft measurement device 100.

[0105] The image correction unit 138 corrects the hull image SG based on the image tilt detected from the hull image SG, and uses the corrected hull image SG as an input image. For example, the image correction unit 138 corrects the hull image SG to a state in which it is projected horizontally or vertically based on the image tilt detected from the hull image SG, and uses the corrected hull image SG as an input image.

[0106] For example, in the above embodiment, the mark detection unit 132 calls the draft mark detection model MD3 using as input the hull image SG itself acquired by the acquisition unit 131. However, the mark detection unit 132 can now call the draft mark detection model MD3 using as input the hull image SG corrected by the image correction unit 138.

[0107] The image correction unit 138 will now be described in more detail. For example, the draft marks MK may be tilted due to the trim of the hull SH, the shooting conditions, etc. If the draft marks MK are tilted, it may be impossible to accurately detect the draft marks MK or accurately read the scale values ​​NM. Therefore, the image correction unit 138 corrects the tilt of the draft marks MK in order to improve processing accuracy. For example, the image correction unit 138 may detect the tilt of the hull image SG using a projection histogram and correct the tilt by stretching the hull image SG in the horizontal or vertical direction.

[0108] As a result, the image correction unit 138 can use the hull image SG including the draft mark MK that has been corrected to be straight for the draft measurement process, thereby improving the accuracy of the draft measurement.

[0109] 7-2. Load Amount Calculation The measurement unit 136 may further estimate the load amount Q of the cargo (e.g., bulk cargo such as grain, ore, or cement) loaded on the hull SH based on the measurement results REx of the draft DR. For example, the measurement unit 136 may compare a combination of the measurement results REx obtained for multiple locations such as the bow, the center of the hull, and the stern with a database in which the load amount is defined for each measurement result REx, thereby estimating the load amount Q on a rule-based basis. Furthermore, the image correction unit 138 may present the load amount Q together with the measurement results REx.

[0110] 8. Hardware Configuration An example of the hardware configuration of a computer corresponding to the draft measurement device according to each of the above-described embodiments (draft measurement device 100 according to the embodiment, draft measurement device 100A according to the modified example) will be described using Fig. 11. Fig. 11 is a block diagram showing an example of the hardware configuration of a computer corresponding to the draft measurement device according to the embodiment.

[0111] 11 , the draft measurement device includes, for example, a processor 801, a ROM 802, a RAM 803, a host bus 804, a bridge 805, an external bus 806, an interface 807, an input device 808, an output device 809, a storage 810, a drive 811, a connection port 812, and a communication device 813. Note that the hardware configuration shown here is an example, and some of the components may be omitted. Furthermore, the draft measurement device may include components other than those shown here.

[0112] (Processor 801) The processor 801 functions, for example, as an arithmetic processing device or control device, and controls the overall operation of each component or part of it based on various programs recorded in the ROM 802, RAM 803, storage 810, or removable recording medium 901.

[0113] (ROM 802, RAM 803) The ROM 802 is a means for storing programs to be read into the processor 801, data to be used for calculations, etc. The RAM 803 temporarily or permanently stores, for example, the programs to be read into the processor 801, and various parameters that change as appropriate when the programs are executed.

[0114] (Host bus 804, bridge 805, external bus 806, interface 807) The processor 801, ROM 802, and RAM 803 are connected to one another via, for example, a host bus 804 that is capable of high-speed data transmission. On the other hand, the host bus 804 is connected to, for example, an external bus 806 that has a relatively low data transmission speed via a bridge 805. Furthermore, the external bus 806 is connected to various components via an interface 807.

[0115] (Input Device 808) Examples of the input device 808 include a mouse, keyboard, touch panel, button, switch, and lever. Furthermore, a remote controller (hereinafter referred to as a remote control) capable of transmitting control signals using infrared rays or other radio waves may also be used as the input device 808. The input device 808 also includes an audio input device such as a microphone.

[0116] (Output Device 809) The output device 809 is a device capable of visually or audibly notifying a user of acquired information, such as a display device such as a CRT (Cathode Ray Tube), LCD, or organic EL, an audio output device such as a speaker or headphones, a printer, a mobile phone, or a facsimile. The output device 809 according to this embodiment also includes various vibration devices capable of outputting tactile stimuli. The output device 809 may also include an AI speaker or a wearable device.

[0117] (Storage 810) The storage 810 is a device for storing various types of data. For example, a magnetic storage device such as a hard disk drive (HDD), a semiconductor storage device, an optical storage device, or a magneto-optical storage device may be used as the storage 810.

[0118] (Drive 811) The drive 811 is a device that reads information recorded on a removable recording medium 901 such as a magnetic disk, optical disk, magneto-optical disk, or semiconductor memory, or writes information to the removable recording medium 901.

[0119] (Connection Port 812) The connection port 812 is a port for connecting an external device 902, such as a USB (Universal Serial Bus) port, an IEEE 1394 port, a SCSI (Small Computer System Interface), an RS-232C port, or an optical audio terminal.

[0120] (Communication device 813) The communication device 813 is a communication device for connecting to a network, such as a communication card for wired or wireless LAN, Bluetooth (registered trademark), or WUSB (Wireless USB), a router for optical communication, a router for ADSL (Asymmetric Digital Subscriber Line), or a modem for various types of communication.

[0121] (Removable recording medium 901) The removable recording medium 901 is, for example, a DVD medium, a Blu-ray (registered trademark) medium, an HD DVD medium, various semiconductor storage media, etc. Of course, the removable recording medium 901 may also be, for example, an IC card equipped with a contactless IC chip, or an electronic device.

[0122] (Externally Connected Device 902) The externally connected device 902 is, for example, a printer, a portable music player, a digital camera, a digital video camera, or an IC recorder.

[0123] In the draft measurement device 100 and the draft measurement device 100A, the memory unit 120 is realized by the ROM 802, the RAM 803, and the storage 810. Also, the control unit 130 realized by the processor 801 reads out and executes each control program (for example, a draft measurement program according to the embodiment) that realizes the acquisition unit 131, the mark detection unit 132, the area detection unit 133, the reading unit 134, the error control unit 135, the measurement unit 136, the presentation unit 137, and the image correction unit 138 from the ROM 802, the RAM 803, etc.

[0124] [9. Other] Of the above processes, all or part of the processes described as being performed automatically may be performed manually. Furthermore, all or part of the processes described as being performed manually may be performed automatically using known methods. In addition, the information, including the processing procedures, specific names, various data, and parameters shown in the above documents and drawings, can be changed as desired unless otherwise specified. For example, the various information shown in each drawing is not limited to the information shown.

[0125] Furthermore, the components of each device shown in the figure are functional concepts and do not necessarily have to be physically configured as shown. In other words, the specific form of distribution and integration of each device is not limited to that shown. Furthermore, all or part of each component may be functionally or physically distributed and integrated in any unit depending on various loads, usage conditions, etc. Furthermore, each of the processes described above may be executed in appropriate combinations within a range that does not contradict each other.

[0126] The above describes in detail the embodiments of the present application based on several drawings, but these are merely examples, and the present invention can be implemented in other forms that include the embodiments described in the Disclosure of the Invention section and that have been modified and improved in various ways based on the knowledge of those skilled in the art.

[0127] 100 Draft measurement device 130 Control unit 131 Acquisition unit 132 Mark detection unit 133 Area detection unit 134 Reading unit 135 Error control unit 136 Measurement unit 137 Presentation unit 138 Image correction unit

Claims

1. A draft measurement program that causes a computer to execute the following steps: an acquisition step for acquiring a hull image in which the hull is photographed; an area detection step for detecting a water surface area in the hull image using a detector that receives the hull image as input; a reading step for reading scale values ​​indicated by draft marks in the hull image using a recognizer that receives the hull image as input; and a measurement step for measuring the draft of the hull based on the detection result of the water surface area and the reading result of the scale values.

2. The draft measurement program described in claim 1, characterized in that the area detection step uses the detector to receive an extracted image in which a predetermined range including the draft mark is extracted from the hull image, to detect the water surface area in the extracted image, and the reading step uses the recognizer to receive the extracted image, to read the scale values ​​indicated by the draft mark in the extracted image.

3. The draft measurement program described in claim 2, characterized in that the hull image is one frame image of a time series of images in which the hull is photographed in chronological order, the area detection step detects a water surface area in each of the extracted images extracted from each of the frame images, the reading step reads a scale value indicated by a draft mark in each of the extracted images, and the measurement step measures the draft of the hull for each of the extracted images and obtains a moving average of the draft measurement results for each of the extracted images as a measurement result to be presented to a user.

4. The draft measurement program according to claim 3, characterized in that the measurement procedure includes excluding, from the draft measurement results for each extracted image, measurement results that are estimated to be outliers, and calculating the moving average using the remaining measurement results.

5. The draft measurement program described in claim 2, characterized in that, when there is a reading result whose reliability does not satisfy a threshold value among the reading results of the scale values ​​for each of the draft marks, the reading procedure corrects the reading result whose reliability does not satisfy a threshold value based on the reading results whose reliability satisfies the threshold value.

6. The draft measurement program according to claim 2, further comprising causing the computer to execute an image correction procedure for correcting the hull image based on an image tilt detected from the hull image and using the corrected hull image as an input image.

7. The draft measurement program according to claim 1, further comprising a display means for displaying to the computer a cause of failure to detect the water surface area or to read the scale value if the detection fails and a solution to the cause is displayed to the user.

8. The draft measurement program according to claim 2, characterized in that the measurement procedure includes determining the coordinates on the extracted image of the intersection between the waterline indicated by the detection result of the water surface area and a straight line tracing the draft marks in a row, and obtaining the reading of the scale value indicated by the draft mark that corresponds to the coordinates as the draft of the hull.

9. The draft measurement program according to claim 8, characterized in that the area detection step generates an image of the water surface portion based on the detection result of the water surface area and the extracted image, and the measurement step identifies the waterline from the image of the water surface portion.

10. A draft measuring device comprising: an acquisition unit that acquires a hull image of a hull; an area detection unit that detects the water surface area in the hull image using a detector that receives the hull image as input; a reading unit that reads scale values ​​indicated by draft marks in the hull image using a recognizer that receives the hull image as input; and a measurement unit that measures the draft of the hull based on the detection result of the water surface area and the reading result of the scale values.

Images