Vessel recognition method for monitoring marine boundary and mobile terminal

Abstract

To provide a vessel recognition method for monitoring a marine boundary and a mobile terminal.SOLUTION: A vessel recognition method for monitoring a marine boundary includes steps for: outputting a real-time camera picture of a sea taken by a camera installed in a mobile terminal S310; transmitting an information request including location information of the mobile terminal to a collection server which collects vessel information in real time in response to output of the real-time camera picture S320; receiving vessel information of a vessel within a preset distance from the location of the mobile terminal from the collection server in response to the information request S330; and fusing the received vessel information with each vessel image of the output real-time camera image and outputting the fusing vessel information by using the received vessel information S340.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to a method for identifying ships and a mobile terminal for maritime border surveillance. [Background technology]

[0002] Maritime boundary monitoring can be said to be a series of processes to ensure the safety of citizens and national security by early detection of abnormal situations at sea, such as ships behaving abnormally, marine accidents, and illegal marine activities, through continuous marine observation. Recent maritime boundary monitoring utilizes a combination of various sensors and systems, including cameras, radar, AIS, V-Pass, and e-Navigation, to enable more accurate and precise maritime boundary monitoring with the development of technology.

[0003] For rapid and accurate maritime border monitoring, it is very important to have a procedure for identifying unidentified vessels through mutual cooperation and information exchange among border monitoring units such as indoor general situation rooms, external coastal areas, and maritime border monitoring vessels.

[0004] Figure 1 illustrates the procedure for identifying unidentified vessels through mutual cooperation between maritime boundary surveillance units.

[0005] Referring to Figure 1, the procedure for identifying an unidentified ship is as follows: while conducting maritime boundary monitoring by each maritime boundary monitoring unit, if an unidentified ship that is difficult for one of the maritime boundary monitoring units to identify appears, the unit sends a ship identification request to another boundary monitoring unit, which identifies the ship and returns the result to the requesting unit, and the unidentified ship is resolved based on the identification result.

[0006] Existing video-based maritime border surveillance systems monitor and identify ships at sea using fixed cameras installed on land or use a separate surveillance system to confirm ships, which requires the installation of many coastal border surveillance cameras and a large number of personnel to monitor them.

[0007] FIG. 2 is a diagram showing an example of a ship recognition screen for each maritime boundary surveillance unit.

[0008] When implementing the procedure for identifying unidentified vessels in the existing maritime boundary monitoring, the location and situational awareness environment are different for each maritime boundary monitoring unit, as shown in Figure 2. Here, the situational awareness environment can include two-dimensional maps, CCTV footage, maritime boundary monitoring vessels, actual coastal areas, etc. Therefore, when exchanging information between maritime boundary monitoring units, mapping of the same vessel, object, topographical feature, etc. is based on verbal description, which is time-consuming and has a high probability of error. [Prior art documents] [Patent documents]

[0009] [Patent Document 1] Republic of Korea Patent Registration No. 10-1703906 (2017.02.01) Summary of the Invention [Problem to be solved by the invention]

[0010] The present invention provides a ship recognition method and a mobile device for maritime boundary monitoring that outputs an augmented reality image that fuses information about a ship that appears in a real-time maritime image captured by a mobile device camera. [Means for solving the problem]

[0011] According to one aspect of the present invention, a vessel recognition method for mobile terminal-based maritime boundary surveillance is disclosed.

[0012] A ship recognition method for maritime boundary monitoring according to an embodiment of the present invention includes the steps of outputting a real-time camera image of the sea taken by a camera installed in the mobile terminal, transmitting an information request including location information of the mobile terminal to a collection server that collects ship information in real time in response to the output of the real-time camera image, receiving ship information of ships within a predetermined distance from the location of the mobile terminal from the collection server in response to the information request, and fusing the ship information with each ship image of the output real-time camera image using the received ship information and outputting the ship information.

[0013] The collection server receives and stores real-time ship information of all ships sailing within pre-defined control areas from the Automatic Identification System (AIS), V-PASS and the High Speed ​​​​Mail-order Radio Communication Network (LTE-M).

[0014] The vessel information includes vessel name, vessel specifications, current date and time, call sign, identification number, position, speed, heading and turn rate.

[0015] The step of fusing and outputting the ship information involves displaying the ship information around each ship image appearing in the real-time camera image using the position, altitude, and PTZ (Pan-Tilt-Zoom) information of the mobile terminal's camera and the ship information collected about ships within a preset distance from the mobile terminal's position.

[0016] The step of combining and outputting the ship information includes displaying the name of the ship around each ship image appearing in the real-time camera image, and when a touch input is received on the ship image, outputting the entire ship information of the touch-inputted ship image through a pop-up window.

[0017] According to another aspect of the invention, a mobile terminal implementing a vessel recognition method for maritime boundary surveillance is disclosed.

[0018] According to an embodiment of the present invention, a mobile terminal includes a memory for storing commands and a processor for executing the commands, and the commands implement a ship recognition method for maritime boundary monitoring, the method including the steps of: outputting real-time camera images of the sea taken by a camera provided in the mobile terminal; transmitting an information request including location information of the mobile terminal to a collection server that collects ship information in real time in response to the output of the real-time camera images; receiving ship information of ships within a preset distance from the location of the mobile terminal from the collection server in accordance with the information request; and fusing the ship information with each ship image in the output real-time camera images using the received ship information and outputting the fusing. Effect of the Invention

[0019] The ship recognition method and mobile device for maritime boundary monitoring according to an embodiment of the present invention can output an augmented reality image that fuses information about a ship that appears in a real-time maritime image captured by the camera of the mobile device. [Brief description of the drawings]

[0020] [Figure 1] This diagram shows the procedure for identifying unidentified vessels through mutual cooperation between maritime boundary monitoring units. [Diagram 2] This figure shows an example of a ship recognition screen by maritime boundary monitoring unit. [Diagram 3] 4 is a flowchart illustrating a ship recognition method for maritime border monitoring performed by a mobile terminal according to an embodiment of the present invention. [Figure 4] FIG. 4 is a diagram for explaining a ship recognition method for maritime boundary monitoring according to the embodiment of the present invention shown in FIG. 3. [Diagram 5] 2 is a diagram illustrating a schematic configuration of a mobile terminal that executes a ship recognition method for maritime boundary monitoring according to an embodiment of the present invention; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0021] As used herein, singular expressions include plural expressions unless otherwise clearly indicated in the context. In this specification, terms such as "comprise" or "include" should not be interpreted as including all of the components or steps described in the specification, but should be interpreted as including some of the components or some of the steps, or including additional components or steps. In addition, terms such as "unit," "module," and the like described in the specification mean a unit that processes at least one function or operation, which may be implemented in hardware or software, or may be implemented in a combination of hardware and software.

[0022] Various embodiments of the present invention will now be described with reference to the accompanying drawings, in which:

[0023] Fig. 3 is a flowchart showing a ship recognition method for maritime boundary monitoring executed by a mobile terminal according to an embodiment of the present invention, and Fig. 4 is a diagram for explaining the ship recognition method for maritime boundary monitoring according to the embodiment of the present invention shown in Fig. 3. Hereinafter, the ship recognition method for maritime boundary monitoring according to the embodiment of the present invention will be explained mainly with reference to Fig. 3 and Fig. 4.

[0024] In step S310, the mobile terminal 100 outputs real-time camera images captured by a camera attached to the mobile terminal 100.

[0025] That is, the mobile terminal 100 can be equipped with a ship recognition application that recognizes a ship from a camera image of the sea. Therefore, the mobile terminal 100 can drive the ship recognition application according to a drive command from the user, start the camera of the mobile terminal 100, and output the camera image via the ship recognition application. At this time, the mobile terminal 100 can output a guidance message to capture an image of the sea where the boundary monitoring is desired in response to the driving of the ship recognition application. This allows the user to check the guidance message and point the camera of the mobile terminal 100 toward the sea where the user wishes to perform surveillance.

[0026] In step S320, the mobile terminal 100 transfers an information request including the location information of the mobile terminal 100 to the collection server 200 that collects ship information in real time according to the output of the real-time camera image.

[0027] Here, the collection server 200 can receive and store in real time ship information of all ships sailing in a preset control area from an Automatic Identification System (AIS), V-PASS, a high-speed maritime radio communication network (LTE-M), etc. Here, the ship information can include ship name, ship specifications, current date and time (date and time), call sign, identification number (MMSI), position (latitude and longitude), speed (SoG), course (CoG), rate of turn, etc.

[0028] In step S330, the mobile terminal 100 receives, in response to the information request forwarded to the collection server 200, ship information of ships within a preset distance from the location of the mobile terminal 100 from the collection server 200.

[0029] That is, the collection server 200 receives a request for information including the location information of the mobile terminal 100, identifies ships within a preset distance from the location of the mobile terminal 100, extracts ship information for the identified ships, and can then forward the information to the requesting mobile terminal 100.

[0030] In step S340, the mobile terminal 100 uses the received ship information to merge the ship information with each ship image in the output real-time camera image and outputs the merged image.

[0031] In other words, the mobile terminal 100 can display ship information of a ship around each ship image appearing in the real-time camera image using the position, altitude, and PTZ (pan-tilt-zoom) information of the camera of the mobile terminal 100, as well as ship information collected about ships within a preset distance from the position of the mobile terminal 100.

[0032] Here, the displayed ship information may be the ship name as shown in Fig. 4. When the mobile terminal 100 receives a touch input on the ship image from the user to view detailed information, the mobile terminal 100 can output the overall ship information of the ship, such as the ship name, ship specifications, current time and date (date and time), call sign, identification number (MMSI), position (latitude and longitude), speed (SoG), course (CoG), rate of turn, etc., through another pop-up window.

[0033] FIG. 5 is a diagram illustrating a schematic configuration of a mobile terminal that executes a ship recognition method for maritime border monitoring according to an embodiment of the present invention.

[0034] Referring to FIG. 5, a mobile terminal according to an embodiment of the present invention includes a processor 10, a memory 20, a communication unit 30, and an interface unit 40.

[0035] The processor 10 may be a CPU or semiconductor device that executes processing instructions stored in a memory 20 .

[0036] The memory 20 may include various types of volatile or non-volatile storage media. For example, the memory 20 may include a ROM, a RAM, and the like.

[0037] For example, memory 20 may store commands for implementing a vessel recognition method for maritime boundary surveillance according to an embodiment of the present invention.

[0038] The communication unit 30 is a means for transmitting and receiving data to and from other devices via a communication network.

[0039] The interface unit 40 can include a network interface for connecting to a network and a user interface.

[0040] On the other hand, the components of the above-mentioned embodiment can be easily understood from a process perspective. That is, each component can be understood as a process. Furthermore, the process of the above-mentioned embodiment can be easily understood from the viewpoint of the components of the device.

[0041] Furthermore, the technical contents described above may be embodied in the form of program instructions that can be executed via various computer means and recorded on a computer-readable medium. The computer-readable medium may include, alone or in combination, program instructions, data files, data structures, and the like. The program instructions recorded on the medium may be specially designed and configured for the embodiments, or may be known and available to those skilled in the art of computer software. Examples of computer-readable recording media include hardware devices specially configured to store and execute program instructions, such as magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, magneto-optical media such as optical disks, read only memory (ROM), random access memory (RAM), and flash memory. Examples of program instructions include not only machine language code, such as produced by a compiler, but also high-level language code that can be executed by a computer using an interpreter, or the like. A hardware device may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

[0042] The above-described embodiments of the present invention have been disclosed for illustrative purposes, and those skilled in the art having ordinary skill in the art may make various modifications, changes, and additions within the spirit and scope of the present invention. These modifications and additions should be considered as falling within the scope of the following claims. [Explanation of symbols]

[0043] 10 Processors 20 Memory 30 Communications Department 40 Interface section 100 mobile devices 200 Collection Server

Claims

1. A ship recognition method for maritime boundary monitoring performed by a mobile terminal, comprising: outputting a real-time camera image of the ocean captured by a camera installed in the mobile terminal; transmitting an information request including location information of the mobile terminal to a collection server that collects ship information in real time according to an output of the real-time camera image; receiving ship information of ships within a preset distance from the location of the mobile terminal from the collection server in response to the information request; and using the received ship information, fusing the ship information with each ship image of the output real-time camera image and outputting the fusing.

2. 2. The method of claim 1, wherein the collection server receives and stores ship information of all ships sailing in a pre-defined control area in real time from an Automatic Identification System (AIS), a V-PASS, and a High Speed ​​​​Marine Communication Network (LTE-M).

3. 2. The method of claim 1, wherein the vessel information includes vessel name, vessel specifications, current date and time, call sign, identification number, position, speed, course and rate of turn.

4. The step of fusing and outputting the ship information includes:

2. The ship recognition method for maritime boundary monitoring as described in claim 1, characterized in that the ship information is displayed around each ship image appearing in the real-time camera image using the position, altitude, and PTZ (Pan-Tilt-Zoom) information of the camera of the mobile terminal and the ship information collected about ships within a preset distance from the position of the mobile terminal.

5. The step of fusing and outputting the ship information includes: Displaying the name of a ship around the image of each ship appearing in the real-time camera image; 5. The method of claim 4, wherein when a touch input is received on the image of the ship, the entire ship information of the touch-inputted image of the ship is output through a pop-up window.

6. A mobile terminal for executing a ship recognition method for maritime border monitoring, A memory for storing commands; a processor for executing the commands; The command: outputting a real-time camera image of the ocean captured by a camera installed in the mobile terminal; transferring an information request including location information of the mobile terminal to a collection server that collects ship information in real time according to an output of the real-time camera image; receiving ship information of ships within a preset distance from the location of the mobile terminal from the collection server in accordance with the information request; and using the received ship information, fusing the ship information with each ship image in the output real-time camera image and outputting the fusing step.

Images