Apparatus and method for automatically constructing marine vessel traffic data set

Abstract

To provide a vessel traffic data set automatic construction device and method for automatically constructing a vessel traffic data set.SOLUTION: A method for automatically constructing a vessel traffic data set, the method comprising: collecting image data related to the sea; detecting an object from the collected image data; collecting numerical / text data related to the sea; and displaying vessel traffic data through an electronic chart using the collected numerical / text data, the method includes extracting an object and nautical chart information on an electronic nautical chart, matching a detected object with the extracted object and nautical chart information, calculating metadata of collected image data using the object and nautical chart information matched with the detected object, mapping and storing the image data and the metadata, and generating a customized data set for a user using the stored image data and metadata.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an apparatus and method for automatically constructing a marine traffic dataset. [Background technology]

[0002] In recent years, artificial intelligence technologies that enable efficient learning even with small datasets have been developed and put to use. However, securing data for AI training and building datasets still require a great deal of time and effort, and improvements are needed.

[0003] Fortunately, in recent years, methods for automatically constructing datasets have been introduced through various research and inventions, but their application to constructing maritime traffic datasets is limited due to the different characteristics of environments and sensors.

[0004] Therefore, there is a need for an automated means of constructing maritime traffic datasets. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Republic of Korea Patent Registration No. 10-1727142 (2017.04.10) Summary of the Invention [Problem to be solved by the invention]

[0006] The present invention aims to provide an apparatus and method for automatically constructing a maritime traffic dataset, which collects video data and numerical / text data, calculates and stores metadata for the video, and generates a user-customized dataset using the stored video and metadata. [Means for solving the problem]

[0007] According to one aspect of the present invention, there is disclosed a method for automatically constructing a marine traffic dataset, which is executed by an automatic marine traffic dataset constructing device.

[0008] A method for automatically constructing a maritime traffic dataset according to an embodiment of the present invention includes the steps of collecting video data related to the sea, detecting objects from the collected video data, collecting numerical / text data related to the sea, displaying maritime traffic data on an electronic nautical chart using the collected numerical / text data and extracting objects and nautical chart information on the electronic nautical chart, matching the detected objects with the extracted objects and nautical chart information, calculating metadata for the collected video data using the objects and nautical chart information matched to the detected objects, mapping and saving the video data and metadata, and generating a user-customized dataset using the saved video data and metadata.

[0009] The step of collecting the video data includes collecting the video data through an optical camera or a thermal camera installed on the ship.

[0010] The process of collecting the numerical / text data involves collecting the numerical / text data through radar (RADAR), automatic identification system (AIS), V-PASS, and LTE-M installed on the ship.

[0011] The process of matching the detected object with the extracted object and nautical chart information involves comparing the position of the object estimated using the video data with the position of the object estimated using the numerical / text data, and identifying the object and nautical chart information corresponding to the object.

[0012] The metadata includes device information, environment information, object detection information, and chart extraction information.

[0013] According to another aspect of the present invention, an automatic marine traffic data set construction device is disclosed.

[0014] An automatic maritime traffic dataset construction device according to an embodiment of the present invention includes a memory for storing commands and a processor for executing the commands, and the commands execute a method for automatically constructing a maritime traffic dataset, including the steps of collecting video data related to the sea, detecting objects from the collected video data, collecting numerical / text data related to the sea, displaying maritime traffic data on an electronic nautical chart using the collected numerical / text data and extracting objects and nautical chart information on the electronic nautical chart, matching the detected objects with the extracted objects and nautical chart information, calculating metadata for the collected video data using the objects and nautical chart information matched to the detected objects, mapping and saving the video data and metadata, and generating a user-customized dataset using the saved video data and metadata. [Effects of the Invention]

[0015] The apparatus and method for automatically constructing a maritime traffic dataset according to an embodiment of the present invention can collect video data and numerical / text data, calculate and store metadata for the video, and generate a user-customized dataset using the stored video and metadata. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a flowchart illustrating an example of a method for automatically constructing a marine traffic dataset executed by an automatic marine traffic dataset construction device according to an embodiment of the present invention. [Figure 2] 2 is a diagram for explaining the method for automatically constructing a maritime traffic dataset according to the embodiment of the present invention shown in FIG. 1; [Figure 3]2 is a diagram for explaining the method for automatically constructing a maritime traffic dataset according to the embodiment of the present invention shown in FIG. 1; [Figure 4] 2 is a diagram for explaining the method for automatically constructing a maritime traffic dataset according to the embodiment of the present invention shown in FIG. 1; [Figure 5] 2 is a diagram for explaining the method for automatically constructing a maritime traffic dataset according to the embodiment of the present invention shown in FIG. 1; [Figure 6] 1 is a diagram illustrating a schematic configuration of a marine traffic dataset automatic construction device according to an embodiment of the present invention; DETAILED DESCRIPTION OF THE INVENTION

[0017] As used herein, singular expressions include plural forms unless the context clearly dictates otherwise. In this specification, terms such as "comprise" or "include" should not be interpreted as including all of the components or steps described in the specification, but may not include some of the components or steps, and may include additional components or steps. Furthermore, terms such as "module" and "module" used in the specification refer to a unit that processes at least one function or operation, and may be implemented by hardware or software, or a combination of hardware and software.

[0018] Various embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0019] Figure 1 is a flowchart illustrating a maritime traffic dataset automatic construction method executed by a maritime traffic dataset automatic construction device according to an embodiment of the present invention, and Figures 2 to 5 are diagrams for explaining the maritime traffic dataset automatic construction method according to the embodiment of the present invention shown in Figure 1. Below, the maritime traffic dataset automatic construction method according to the embodiment of the present invention will be explained mainly with reference to Figure 1, and Figures 2 to 5 will also be referred to.

[0020] In step S110, the automatic maritime traffic data set construction device collects maritime image data.

[0021] For example, the automatic construction device for marine traffic datasets can collect video data related to the sea through cameras such as optical cameras and thermal imaging cameras installed on ships, as shown in FIG.

[0022] In step S120, the automatic maritime traffic dataset construction device detects objects from the collected video data.

[0023] For example, the marine traffic dataset automatic construction device can detect objects such as ships from collected video data, as shown in FIG.

[0024] In step S130, the automatic maritime traffic dataset construction device collects numerical / text data related to the sea.

[0025] For example, as shown in Figure 4, the automatic maritime traffic dataset construction device can collect maritime-related numerical and text data, such as maritime traffic data of ships at sea, through radar (RADAR), Automatic Identification System (AIS), V-PASS, LTE-M, etc. installed on ships.

[0026] In step S140, the automatic maritime traffic dataset construction device displays maritime traffic data on an electronic nautical chart using the collected numerical / text data, and extracts objects and chart information from the electronic nautical chart.

[0027] In step S150, the automatic maritime traffic dataset construction device matches the detected object with the extracted object and chart information on the electronic chart, and expands the object information.

[0028] In other words, the automatic maritime traffic dataset construction device can find the object and nautical chart information corresponding to the object by comparing the position of the object estimated using video data with the position of the object estimated using numerical / text data.

[0029] For example, as shown in Figure 5, the automatic maritime traffic dataset construction device can precisely estimate the three-dimensional coordinates of each object in the video using information such as the video shooting position and direction, the equipment's attitude (PTZ), focal length, and equipment parameters, convert the estimated three-dimensional coordinates of each object into two-dimensional nautical chart coordinates, and compare the converted two-dimensional nautical chart coordinates with the coordinates of objects collected and estimated from radar (RADAR), automatic identification system (AIS), V-PASS, LTE-M, electronic nautical charts, etc., thereby matching the detected objects with the extracted object and nautical chart information.

[0030] In [Step S160], the automatic maritime traffic dataset construction device calculates metadata for the collected video data using objects and nautical chart information matched to the detected objects, and maps and stores the video data and metadata.

[0031] For example, metadata can refer to information necessary for artificial intelligence learning among collected data, such as device information, environmental information, object detection information, and chart extraction information.

[0032] In step S170, the automatic maritime traffic dataset construction device generates a user-customized dataset using the stored video data and metadata.

[0033] For example, if a user wants to generate a dataset for video-based object detection and ship type classification, they can select ship type information from the metadata mapped to the video data, and save and use it in accordance with the input format for artificial intelligence learning.

[0034] FIG. 6 is a diagram showing a schematic example of the configuration of a marine traffic dataset automatic construction device according to an embodiment of the present invention.

[0035] Referring to FIG. 6, the automatic construction device for marine traffic datasets according to the embodiment of the present invention includes a processor (10), a memory (20), a communication unit (30) and an interface unit (40).

[0036] The processor (10) may be a CPU or semiconductor device that executes processing commands stored in the memory (20).

[0037] The memory 20 may include various types of volatile or non-volatile storage media, such as ROM, RAM, etc.

[0038] For example, the memory 20 may store commands for executing a method for automatically constructing a marine traffic dataset according to an embodiment of the present invention.

[0039] The communication unit (30) is a means for transmitting and receiving data to and from other devices via a communication network.

[0040] The interface unit (40) can include a network interface for connecting to a network and a user interface.

[0041] Meanwhile, the components of the above-described embodiments can be easily understood in terms of processes. That is, each component can be understood as a process. Furthermore, the processes of the above-described embodiments can be easily understood in terms of device components. Furthermore, the above-described technical content can be realized in the form of program commands executable by various computer means and recorded on a computer-readable medium. The computer-readable medium can include program commands, data files, data structures, etc., alone or in combination. The program commands recorded on the medium can be specially designed and configured for the embodiments, or they can be well-known and available in the field of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specially configured to store and execute program commands, such as ROMs, RAMs, and flash memories. Examples of program commands include machine code generated by a compiler and code in a high-level language executed by a computer using an interpreter, etc. A hardware device can be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

[0042] The above-described embodiments of the present invention have been disclosed for illustrative purposes, and those skilled in the art having ordinary skill in the art may make various modifications, changes, and additions within the spirit and scope of the present invention, and these modifications, changes, and additions should be construed as falling within the scope of the following claims. [Explanation of symbols]

[0043] 10 processors 20 memory 30 Communications Department 40 Interface section

Claims

1. In the automatic construction method of a maritime traffic dataset executed by the automatic construction device of a maritime traffic dataset, collecting video data about the sea; detecting an object from the collected video data; collecting said numerical / textual data relating to the sea; displaying marine traffic data on an electronic nautical chart using the collected numerical / text data, and extracting objects and chart information on the electronic nautical chart; matching the detected objects with extracted object and chart information; calculating metadata of the collected image data using the objects and nautical chart information matched to the detected objects, and mapping and storing the image data and metadata; and A method for automatically constructing a maritime traffic dataset, comprising the step of generating a customized dataset for a user using the stored video data and metadata.

2. 2. The method of claim 1, wherein the step of collecting the image data comprises collecting the image data through an optical camera or a thermal imaging camera installed on a ship.

3. 2. The method of claim 1, wherein the step of collecting the numerical / text data comprises collecting the numerical / text data through a radar, an automatic identification system (AIS), a V-PASS, or LTE-M installed on a ship.

4. 2. The method of claim 1, wherein the step of matching the detected object with the extracted object and nautical chart information includes comparing a position of the object estimated using the video data with a position of the object estimated using the numerical / text data to find an object and nautical chart information corresponding to the object.

5. The method for automatically constructing a marine traffic dataset according to claim 1, wherein the metadata includes equipment information, environment information, object detection information, and nautical chart extraction information.

6. In the automatic construction device for marine traffic datasets, memory for storing commands; and a processor for executing the commands; The command collecting video data about the sea; detecting an object from the collected video data; collecting said numerical / textual data relating to the sea; displaying marine traffic data on an electronic nautical chart using the collected numerical / text data, and extracting objects and chart information on the electronic nautical chart; matching the detected objects with the extracted object and nautical chart information; calculating metadata of the collected image data using the objects and nautical chart information matched to the detected objects, and mapping and storing the image data and metadata; and The apparatus for automatically constructing a maritime traffic dataset is characterized by performing a step of generating a customized dataset for a user using the stored video data and metadata.

Images