Method and apparatus for providing ship navigation information interface

Abstract

The present disclosure relates to a method and apparatus for providing a ship navigation information interface. A method for providing a ship navigation information interface according to one embodiment disclosed herein may receive data collected by a sensor mounted on a ship and data related to a navigation mode of the ship, and generate an interface on the basis of the data collected by the sensor and the data related to the navigation mode.

Description

Method and device for providing a vessel operation information interface

[0001] The present invention relates to a method and apparatus for providing a ship operation information interface.

[0002] Typically, users of small vessels steer and throttle to berth or dock. However, due to the inexperience of users of recreational vessels and the influence of environmental disturbances such as currents and winds, berthing or docking in confined spaces can be challenging. Furthermore, vessel accidents most frequently occur during berthing or docking, making it difficult for the general public to easily operate or control recreational vessels.

[0003] Accordingly, the development of technology that enables ships to automatically dock or berth at precise locations is actively underway.

[0004] The background technology described above is technical information that the inventor possessed for the purpose of deriving the present invention or acquired in the process of deriving the present invention, and cannot necessarily be considered as publicly known technology disclosed to the general public prior to the application for the present invention.

[0005] The purpose of the present disclosure is to provide a method and device for providing a vessel operation information interface. The problems addressed by the present disclosure are not limited to those mentioned above. Other problems and advantages of the present disclosure not mentioned above can be understood through the following description and will be more clearly understood through the embodiments of the present disclosure. Furthermore, it will be appreciated that the problems and advantages addressed by the present disclosure can be realized by the means and combinations thereof set forth in the claims.

[0006] A first aspect of the present disclosure provides a method for providing a vessel operation information interface, comprising: receiving data collected by a sensor mounted on a vessel and data relating to an operation mode of the vessel; and generating an interface based on the data collected by the sensor and the data relating to the operation mode; wherein the interface comprises a vessel graphic interface configured to display dynamic graphic elements relating to the vessel and objects surrounding the vessel; and a map interface configured to display a map relating to a location of the vessel; and a method for changing based on the operation mode.

[0007] A second aspect of the present disclosure is a device providing a ship operation information interface, comprising: a memory having at least one program stored therein; and a processor operating by executing the at least one program; wherein the processor receives data collected by a sensor mounted on a ship and data regarding an operation mode of the ship, and generates an interface based on the data collected by the sensor and the data regarding the operation mode, wherein the interface comprises: a ship graphic interface configured to display three-dimensional dynamic graphic elements regarding the ship and objects surrounding the ship; a map interface configured to display a map regarding the location of the ship; and

[0008] A device may be provided that includes at least a portion of a surround view interface including an individual camera view controller configured to individually check image data collected by each of one or more cameras mounted on the vessel, and that changes based on the operating mode.

[0009] A third aspect of the present disclosure can provide a computer-readable recording medium having recorded thereon a program for executing the method according to the first aspect on a computer.

[0010] According to various embodiments of the present disclosure, an interface may be provided that allows a user of a ship on board to easily obtain information about the ship and information related to the operation of the ship.

[0011] In particular, users can control ships and their operations by interacting with the provided interface, thereby increasing safety and convenience.

[0012] Figure 1 is a conceptual diagram for explaining the autonomous navigation system of the present disclosure.

[0013] FIG. 2 is a conceptual diagram illustrating an interface device according to one embodiment of the present disclosure.

[0014] FIG. 3 is a conceptual diagram for explaining an operation mode according to one embodiment of the present disclosure.

[0015] FIG. 4 is a diagram illustrating an interface according to one embodiment of the present disclosure.

[0016] FIG. 5 is a diagram illustrating in detail a ship graphic interface according to one embodiment of the present disclosure.

[0017] FIG. 6 is a diagram illustrating in detail a ship graphic interface according to another embodiment of the present disclosure.

[0018] FIGS. 7A to 7E are diagrams illustrating a ship graphic interface during a process in which automatic docking is performed according to one embodiment of the present disclosure.

[0019] FIGS. 8A and 8B are diagrams illustrating a customizable interface according to one embodiment of the present disclosure.

[0020] FIG. 9 is a diagram illustrating a customizable interface according to another embodiment of the present disclosure.

[0021] FIG. 10 is a flowchart of a method according to one embodiment of the present disclosure.

[0022] FIG. 11 is a block diagram of a device according to one embodiment of the present disclosure.

[0023] According to one embodiment of the present invention for solving the above technical problem, a method for providing a ship operation information interface comprises: a step of receiving data collected by a sensor mounted on a ship and data regarding an operation mode of the ship; and a step of generating an interface based on the data collected by the sensor and the data regarding the operation mode; wherein the interface comprises a ship graphic interface configured to display dynamic graphic elements regarding the ship and objects surrounding the ship; and a map interface configured to display a map regarding the location of the ship; and the method changes based on the operation mode.

[0024] In the above method, the ship graphic interface may display a driving area indicating an area in which the ship can drive in an auto mode; a planned route set for the ship to move; and a future route which is a route that the ship is expected to move in the future.

[0025] In the above method, the map interface can be switched to a surround view interface based on whether the operating mode is one of the automatic operating modes.

[0026] In the above method, the surround view interface may include an individual camera view controller configured to individually check image data collected by each of one or more cameras mounted on the ship.

[0027] In the above method, the individual camera view controller may include a shape designed in consideration of the spacing and viewing angle of the cameras installed on the ship.

[0028] In the above method, the individual camera view controller can further display the screen of the camera corresponding to the shape as a sub-screen through the shape.

[0029] In the above method, the ship graphic interface may include a speed information object, an operating mode information object, a fuel information object, or a depth information object.

[0030] In the above method, the ship graphic interface includes a heading angle maintenance mode initiation object, a position maintenance mode initiation object, or a path tracking mode initiation object, and based on receiving an input for interacting with the heading angle maintenance mode initiation object, the navigation mode can be switched to the heading angle maintenance mode, based on receiving an input for interacting with the position maintenance mode initiation object, the navigation mode can be switched to the position maintenance mode, and based on receiving an input for interacting with the path tracking mode initiation object, the navigation mode can be switched to the path tracking mode.

[0031] In the above method, the method may include: a step of setting a berthing position corresponding to an input based on receiving an input for interacting with a graphic element regarding a berthing position while a graphic element regarding a berthing position is displayed on the ship graphic interface; and a step of initiating automatic berthing based on the set berthing position.

[0032] In the above method, the ship graphic interface may include a berthing stage display object, and the method may further include a step of detecting a berthing stage corresponding to the current ship among berthing stages in the automatic berthing mode based on initiating the automatic berthing; and a step of generating the berthing stage display object based on the berthing stage.

[0033] In the above method, if the berthing step is an approaching step in which the vessel approaches the berth, the vessel graphic interface may display a future route and a berthing route as the berthing step display object, wherein the future route may be a route along which the vessel is expected to move at a future point in time, and the berthing route may be a route along which the vessel is expected to move while berthing.

[0034] In the above method, if the berthing step is a positioning step for determining a berthing position at a berth, the ship graphic interface can display a process in which the surrounding environment including the berth rotates based on the ship.

[0035] In the above method, if the berthing step is a pushing step in which the ship enters the berth, the ship graphic interface may include an object indicating the direction of pushing control as the berthing step display object.

[0036] In the above method, if the berthing step is a completion step in which the berthing of the ship is completed, the ship graphic interface may display an anchoring display object in the shape of concentric circles at the location where the berthing of the ship is completed.

[0037] In the above method, the ship graphic interface is generated according to one of a third-person viewpoint and a viewpoint viewing the ship from the air and includes a viewpoint state object, and the method may further include a step of switching the ship graphic interface from the third-person viewpoint to a viewpoint viewing the ship from the air, or from the viewpoint viewing the ship from the air to the third-person viewpoint, based on receiving an input interacting with the viewpoint state object.

[0038] In the above method, the ship graphic interface may further include a step of changing the area of ​​the ship graphic interface and the map interface based on receiving an input that interacts with the size adjustment object, wherein the ship graphic interface includes a size adjustment object.

[0039] According to another embodiment of the present invention for solving the above technical problem, a device for providing a ship operation information interface comprises: a memory having at least one program stored therein; and a processor operating by executing the at least one program; wherein the processor receives data collected by a sensor mounted on a ship and data regarding a ship operation mode, and generates an interface based on the data collected by the sensor and the data regarding the ship operation mode, and the interface includes at least a part of a ship graphic interface configured to display three-dimensional dynamic graphic elements regarding the ship and objects around the ship; a map interface configured to display a map regarding the location of the ship; and a surround view interface including individual camera view controllers configured to individually check image data collected by each of one or more cameras mounted on the ship, and can change based on the operation mode.

[0040] One embodiment of the present invention can provide a computer-readable recording medium having recorded thereon a program for executing the above method on a computer.

[0041] The advantages and features of the present invention, and the methods for achieving them, will become clearer with reference to the embodiments described in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments presented below, but can be implemented in various different forms, and it should be understood that it includes all transformations, equivalents, and substitutes included in the spirit and technical scope of the present invention. The embodiments presented below are provided to ensure that the disclosure of the present invention is complete and to fully inform those skilled in the art of the invention of the scope of the invention. In describing the present invention, if a detailed description of a related known technology is judged to obscure the gist of the present invention, the detailed description thereof will be omitted.

[0042] The terminology used in this application is only used to describe specific embodiments and is not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly indicates otherwise. In this application, it should be understood that the terms "comprise" or "have" indicate the presence of a feature, number, step, operation, component, part, or combination thereof described in the specification, but do not preclude the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0043] Some embodiments of the present disclosure may be represented by functional block configurations and various processing steps. Some or all of these functional blocks may be implemented by various hardware and / or software configurations that perform specific functions. For example, the functional blocks of the present disclosure may be implemented by one or more microprocessors or by circuit configurations for a given function. Furthermore, for example, the functional blocks of the present disclosure may be implemented in various programming or scripting languages. The functional blocks may be implemented by algorithms that execute on one or more processors. Furthermore, the present disclosure may employ conventional techniques for electronic configuration, signal processing, and / or data processing. Terms such as "mechanism," "element," "means," and "configuration" may be used broadly and are not limited to mechanical and physical configurations.

[0044] Additionally, the connecting lines or connecting members between components depicted in the drawings are merely exemplary representations of functional connections and / or physical or circuit connections. In an actual device, connections between components may be represented by various functional, physical, or circuit connections that may be replaced or added.

[0045] Additionally, terms including ordinal numbers, such as "first" or "second," used in the specification may be used to describe various components, but the components should not be limited by the terms. The terms may be used to distinguish one component from another.

[0046] Figure 1 is a conceptual diagram for explaining an autonomous navigation system according to the present disclosure.

[0047] Referring to FIG. 1, the autonomous navigation system (100) may include a ship's control device (110), an EIU (Engine Interface Unit 120), an Autonomous Navigation Processor (130, hereinafter, an autonomous navigation processing unit), and an engine (140).

[0048] The steering device (110) may include at least a portion of a throttle lever, a steering wheel, and a joystick. However, the present invention is not limited thereto, and the steering device (110) may include other ship devices. The steering device (110) may be referred to as a helm station according to an embodiment.

[0049] The EIU (120) may refer to a device that acquires a signal (S1) from a steering device (110) included in a ship through a communication network within the ship and transmits it to the engine (140). The signal (S1) may include a message or protocol of several steering devices (110) included in the ship. The EIU (120) may transmit the signal (S1) of the steering device (110) as is to the engine (140), or may convert the signal (S1) of the steering device (110) and then transmit the converted signal (S2) to the engine (140). The present invention is not limited thereto.

[0050] The EIU (120) may be a device that enables switching between the autonomous navigation mode (or automatic navigation mode) and the manual navigation mode of the ship by transmitting or injecting a signal (S1) of the steering device (110). The EIU (120) may determine whether the current navigation mode of the ship is the autonomous navigation mode or the manual navigation mode based on a control command received from the autonomous navigation processing device (130). The EIU (120) may also receive the navigation status of the ship determined from the autonomous navigation processing device (130). In this case, the EIU (120) may operate based on the determined navigation status. According to an embodiment, the EIU (120) may determine the autonomous navigation mode by detecting a control value output from the steering device (110) even when the user does not control the steering devices (110).

[0051] The EIU (120) can be connected to the ship's steering device (110) and engine (140) via an internal communication network. The internal communication network can include a CAN (Controller Area Network). Here, CAN can refer to an internal communication network of a ship, automobile, etc. that can perform data transmission between ECUs (Engine Control Units), control of various steering devices (110), control of a system, etc., and is not limited thereto. The internal communication network can refer to any communication network that can transmit data between the ship's steering device (110) and the engine (140). According to an embodiment, a user can check the status of the linkage between the steering devices (110), the autonomous navigation processing unit (130), and the engine (140) by using the EIU (120).

[0052] The autonomous navigation processing unit (130) may be a device that processes control commands for controlling the ship's steering device (110) when the ship is in autonomous navigation mode. In other words, even when the user does not operate the steering device (110), the autonomous navigation processing unit (130) may generate commands for controlling the ship's steering device (110). In addition, the autonomous navigation processing unit (130) may transmit the ship's navigation status to the EIU (120) so that the EIU (120) may operate based on the ship's navigation status.

[0053] The engine (140) may be a device that operates based on control commands from the ship's steering devices (110). For example, when the ship is in manual navigation mode, the engine (140) may operate based on user input operating the steering device (110). As another example, when the ship is in autonomous navigation mode, the engine (140) may operate based on control commands received from the autonomous navigation processing device (130).

[0054] Meanwhile, the method for providing a vessel operation information interface according to various embodiments of the present disclosure may be understood to be performed by a device providing a vessel operation information interface (hereinafter, "device providing an interface"). The device providing the interface of the present disclosure may be the autonomous navigation processing device (130) of FIG. 1 or may be understood to be a part of the autonomous navigation processing device (130) of FIG. 1.

[0055] FIG. 2 is a conceptual diagram illustrating an interface device according to one embodiment of the present disclosure.

[0056] In one embodiment, the autonomous navigation processing device (130) may be connected to an input interface device (201) and an output interface device (202).

[0057] An input interface device (201) may refer to a device or component that allows a user to transmit commands or data to a ship or autonomous navigation processing device (130). The user can cause the ship to perform an operation or input information through the input interface device (201).

[0058] The input interface device (201) may be implemented based on any method, such as a physical input device, a voice input device, a sensor-based input device, a virtual input device, or a special input device, to receive user input. For example, the input interface device (201) may include a switch, a touchscreen, a joystick, a mouse, a keyboard, a microphone, a voice recognition device, a motion sensor, a fingerprint recognition device, a camera, a gesture recognition device, an AR / VR controller, or the like.

[0059] An output interface device (202) may refer to a device or component that transmits data processed by a ship or autonomous navigation processing device (130) in a form understandable to a user. The user can obtain information related to the operation of the ship through the output interface device (202).

[0060] The output interface device (202) may be implemented based on any method, such as a visual output device, an auditory output device, a tactile output device, or a physical output device, to output data. For example, the output interface device (202) may include a display device, a speaker, a vibration motor, a printer, an AR / VR output device, or the like.

[0061] The interface according to various embodiments of the present disclosure may be displayed through an output interface device (202).

[0062] Meanwhile, in FIG. 2, the autonomous navigation processing device (130), the input interface device (201), and the output interface device (202) are each illustrated as separate configurations, but at least some of the autonomous navigation processing device (130), the input interface device (201), and the output interface device (202) may be implemented as a single device. For example, the input interface device (201) and the output interface device (202) may be included in the autonomous navigation processing device (130).

[0063] In the present disclosure, a device providing an interface can generate an interface based on data collected by a sensor and data regarding an operating mode. The generated interface may be a graphical interface.

[0064] In the present disclosure, generating an interface may include estimating, calculating, or reproducing data based on collected data. For example, a device providing an interface may estimate an object based on data collected by an onboard sensor.

[0065] For example, a device providing an interface can estimate a static object using a lidar sensor. For example, the device providing the interface can acquire point data of an object using the lidar sensor and divide the acquired point data into at least one region. Furthermore, the device providing the interface can determine representative point data for each divided region and estimate the static object using the determined representative point data.

[0066] For example, a device providing an interface can estimate a dynamic object using a lidar sensor and a camera. For example, the device providing an interface can acquire point data of an object using a lidar sensor and image data of the object using a camera. Furthermore, the device providing an interface can generate integrated information by integrating the respective data by correlating the point data of the object and the image data of the object. Furthermore, the device providing an interface can perform clustering based on the integrated information to determine a representative cluster and estimate the dynamic object using the determined representative cluster.

[0067] A device providing a ship operation information interface of the present disclosure can receive data regarding an operation mode and generate an interface based on the data regarding the operation mode.

[0068] FIG. 3 is a conceptual diagram for explaining an operation mode according to one embodiment of the present disclosure.

[0069] In the present disclosure, the ship's operating mode can be switched based on user input, the ship's operating status, environmental conditions, etc. In the present disclosure, a device providing an interface can detect a switch in the ship's operating mode and determine and generate an interface based on the switched operating mode of the ship.

[0070] In one embodiment, the vessel's operating mode may be broadly categorized into either a manual operating mode or an automatic operating mode. Manual operating mode refers to a mode in which vessel control is performed based on user input, while automatic operating mode refers to a mode in which at least part of the vessel's control is performed without user input.

[0071] In one embodiment, the autonomous navigation mode may include a position-holding mode, a heading angle-holding mode, a path-following mode, and an automatic berthing mode. That is, when the navigation mode of the vessel is an autonomous navigation mode, the navigation mode of the vessel may be any one of the position-holding mode, the heading angle-holding mode, the path-following mode, and the automatic berthing mode.

[0072] In the present disclosure, the operating mode of the vessel can be switched from any one of a manual operating mode, a position holding mode, a heading angle holding mode, a path following mode, and an automatic berthing mode to any other one.

[0073] In this disclosure, position-holding mode may refer to a mode in which a vessel is controlled to maintain a specific location. The specific location may be defined by latitude and longitude. In one embodiment, the specific location may be set by the user. In another embodiment, the specific location may be set as the vessel's location when position-holding mode is initiated.

[0074] In the present disclosure, the heading angle maintenance mode may refer to a mode in which the heading angle of a vessel is controlled to be maintained. The heading angle of a vessel may refer to the angle formed by a meridian passing through the vessel and a line defined as bow-stern. In one embodiment, the heading angle of the vessel may be set by the user. In another embodiment, the heading angle of the vessel may be set to the heading angle of the vessel when the heading angle maintenance mode is initiated.

[0075] In the present disclosure, the path tracking mode may refer to a mode in which the vessel is controlled to follow a specific path. The specific path may be preset by the user.

[0076] In the present disclosure, the automatic berthing mode may refer to a mode in which a vessel is automatically controlled to berth at a berthing location. In one embodiment, the berthing location may be set by the user. In another embodiment, the berthing location may be preset.

[0077] In one embodiment, each operating mode may include one or more states. The states of an operating mode may indicate the operating mode's execution status.

[0078] In the present disclosure, data regarding the operation mode of a vessel may be understood to include both the type of operation mode and the state of the operation mode.

[0079] In one embodiment, the state of the manual operation mode may be either a running state or an emergency state.

[0080] In one embodiment, the state of the position hold mode may consist solely of the running state.

[0081] In one embodiment, the state of the player angle maintenance mode may consist of only the running state.

[0082] In one embodiment, the state of the path tracing mode can be any one of a running state, a stopped state, and a completed state.

[0083] In one embodiment, the state of the automatic eye contact mode can be any one of a running state, a stopped state, and a completed state.

[0084] FIG. 4 is a diagram illustrating an interface according to one embodiment of the present disclosure.

[0085] The interface illustrated in Fig. 4 may be a main interface. A main interface may refer to an interface that can be created and displayed without additional user settings.

[0086] In one embodiment, the interface (400) may include a ship graphic interface (410) and a map interface (420). The ship graphic interface (410) may be an interface configured to display dynamic graphic elements relating to the ship and objects surrounding the ship. The map interface (420) may be an interface configured to display a map relating to the location of the ship.

[0087] Referring to FIG. 4, a ship graphic interface (410) configured to display dynamic graphic elements relating to the ship and objects surrounding the ship and a map interface (420) configured to display a map relating to the location of the ship are illustrated.

[0088] In one embodiment, the vessel graphic interface (410) may include a driving area (411), a planned route (412), and a future route (413). The driving area (411) refers to an area in which the vessel can operate in Auto Mode. For example, if the vessel is outside the driving area (411), the Auto Mode cannot be activated. In FIG. 4, the planned route (412) refers to a route that has been preset (planned) for the vessel to travel. The future route (413) refers to a route that the vessel is expected to travel in the future, reflecting the vessel's current status / location. Depending on the embodiment, the vessel graphic interface (410) may be displayed in three dimensions. The vessel graphic interface (410) according to various embodiments of the present disclosure will be described in detail below.

[0089] In one embodiment, the map interface (420) may include a map of the vessel's location. The map of the vessel's location may be generated based on the vessel's location data. The map of the vessel's location allows the user to determine the vessel's location and route.

[0090] In one embodiment, the map interface (420) may include a route display object. The route display object may be displayed on a map regarding the vessel's location. The route display object may include one or more waypoints. The one or more waypoints may be preset or user-defined. In one embodiment, the user may request additional actions by interacting with the route display object. For example, the device providing the interface or the autonomous navigation processing device (130) may switch the vessel's navigation mode to a route tracking mode based on receiving input interacting with the route display object.

[0091] In one embodiment, the map interface (420) may include a charity object. The charity object may be displayed on the map relative to the vessel's location. The charity object may be generated based on the vessel's current location and the vessel's heading angle.

[0092] In one embodiment, the map interface (420) may include a navigation information panel. The navigation information panel may be configured to display information related to the vessel's operation. In one embodiment, the navigation information panel may include information regarding the expected arrival time and remaining distance. In one embodiment, the navigation information panel may include an object for enabling the function of saving a specific route as a favorite.

[0093] In one embodiment, the map interface (420) may include one or more objects to enhance the user's convenience, such as an object with which a user can interact to check notifications, an object with which a user can interact to set the environment of a vessel or interface device, an object with which a user can interact to check navigational notices or warning messages, an object with which a user can interact to place pins on the map interface, and an object with which a user can interact to zoom in or out on the map.

[0094] FIG. 5 is a diagram illustrating in detail a ship graphic interface according to one embodiment of the present disclosure.

[0095] Referring to FIG. 5, a ship graphic interface (500) is illustrated, and the ship graphic interface (500) may correspond to the ship graphic interface (410) described above with reference to FIG. 4.

[0096] In one embodiment, the ship graphic interface (500) may include a dynamic graphic interface (510) configured to display dynamic graphic elements relating to the ship and objects surrounding the ship.

[0097] Referring to FIG. 5, graphic elements relating to a ship included in a dynamic graphic interface (510), graphic elements relating to a berth located around the ship, and graphic elements relating to a berth location located around the ship (520) are illustrated. The graphic elements included in the dynamic graphic interface (510) may dynamically change as the ship moves or objects around the ship move.

[0098] The process of creating a dynamic graphic element or dynamic graphic interface (510) of the present disclosure will be described in detail later.

[0099] In one embodiment, a user of the vessel may interact with a graphical element (520) regarding a berthing location. In one embodiment, the user of the vessel may set a berthing location by interacting with the graphical element (520) regarding a berthing location while the graphical element (520) regarding a berthing location is displayed on the vessel graphical interface (500). In one embodiment, the autonomous navigation processing device (130) may set a berthing location corresponding to the input based on receiving an input for interacting with the graphical element (520) regarding a berthing location, and may initiate automatic berthing based on the set berthing location.

[0100] In one embodiment, the ship graphic interface (500) may include a speed information object (501). The speed information object (501) may be an object for indicating the speed of the ship.

[0101] Referring to FIG. 5, a speed information object (501) is illustrated. The location of the speed information object (501) within the ship graphic interface (500) may vary depending on the user's settings.

[0102] In one embodiment, the ship graphic interface (500) may include an operation mode information object (502). The operation mode information object (502) may be an object for indicating the operation mode of the ship.

[0103] Referring to FIG. 5, an operation mode information object (502) is illustrated. The location of the operation mode information object (502) within the ship graphic interface (500) may vary depending on the user's settings.

[0104] In one embodiment, the vessel graphical interface (500) may include an autonomous navigation status object (503). The autonomous navigation status object (503) may be an object indicating whether the vessel is controlled by autonomous navigation. If the vessel's navigation mode is autonomous navigation mode, the autonomous navigation status object (503) may be activated.

[0105] Referring to FIG. 5, an automatic navigation status object (503) is illustrated. The location of the automatic navigation status object (503) within the ship graphic interface (500) may vary depending on the user's settings.

[0106] In one embodiment, the vessel graphical interface (500) may include a heading angle maintenance mode initiation object (504). In one embodiment, a user of the vessel may interact with the heading angle maintenance mode initiation object (504). In one embodiment, the autonomous navigation processing unit (130) may switch the vessel's navigation mode to the heading angle maintenance mode based on receiving input interacting with the heading angle maintenance mode initiation object (504).

[0107] Referring to FIG. 5, a head angle maintenance mode initiation object (504) is illustrated. The location of the head angle maintenance mode initiation object (504) within the ship graphic interface (500) may vary depending on the user's settings.

[0108] In one embodiment, the vessel graphical interface (500) may include a position hold mode initiation object (505). In one embodiment, a user of the vessel may interact with the position hold mode initiation object (505). In one embodiment, the autonomous navigation processing unit (130) may switch the vessel's navigation mode to position hold mode based on receiving input interacting with the position hold mode initiation object (505).

[0109] Referring to FIG. 5, a position maintenance mode initiation object (505) is illustrated. The location of the position maintenance mode initiation object (505) within the ship graphic interface (500) may vary depending on the user's settings.

[0110] In one embodiment, the ship graphical interface (500) may include a viewpoint state object (506). The viewpoint state object (506) may be an object for indicating a viewpoint of the dynamic graphical interface (510) or the ship graphical interface (500). For example, the viewpoint state object (506) may indicate whether the dynamic graphical interface (510) is generated and displayed based on either a third-person viewpoint or an aerial viewpoint of the ship.

[0111] In one embodiment, a user of the vessel may interact with a viewpoint state object (506). In one embodiment, the autonomous navigation processing unit (130) may generate a dynamic graphical interface (510) by switching viewpoints based on receiving input interacting with the viewpoint state object (506). For example, based on input interacting with the viewpoint state object (506), the viewpoint of the dynamic graphical interface (510) may switch from a third-person viewpoint, to a viewpoint looking at the vessel from above, or from a viewpoint looking at the vessel from above to a third-person viewpoint.

[0112] Referring to FIG. 5, a viewpoint state object (506) is illustrated. The location of the viewpoint state object (506) within the ship graphic interface (500) may vary depending on the user's settings.

[0113] In one embodiment, the vessel graphical interface (500) may include a path tracking mode initiation object (507). In one embodiment, a user of the vessel may interact with the path tracking mode initiation object (507). In one embodiment, the autonomous navigation processing unit (130) may switch the vessel's navigation mode to the path tracking mode based on receiving input interacting with the path tracking mode initiation object (507).

[0114] Referring to FIG. 5, a path tracking mode initiation object (507) is illustrated. The location of the path tracking mode initiation object (507) within the ship graphic interface (500) may vary depending on the user's settings.

[0115] In one embodiment, the ship graphical interface (500) may include a fuel information object (508). The fuel information object (508) may be an object for indicating the remaining amount of fuel.

[0116] Referring to FIG. 5, a fuel information object (508) is illustrated. The location of the fuel information object (508) within the ship graphic interface (500) may vary depending on the user's settings.

[0117] In one embodiment, the ship graphic interface (500) may include a depth information object (509). The depth information object (509) may be an object for indicating the depth of the water, which is the distance from the seabed to the water surface.

[0118] Referring to FIG. 5, a depth information object (509) is illustrated. The location of the depth information object (509) within the ship graphic interface (500) may vary depending on the user's settings.

[0119] Below, the process of creating a dynamic graphic element or dynamic graphic interface (510) is described.

[0120] In one embodiment, the autonomous navigation processing device (130) may obtain information about objects around the vessel and generate dynamic graphic elements based on the obtained information about the objects.

[0121] Specifically, in one embodiment, information about objects in the vicinity of the vessel may be collected by sensors including cameras mounted on the vessel.

[0122] In one embodiment, the autonomous navigation processing unit (130) can create a grid centered on the charity and place objects in the grid based on information about the acquired objects.

[0123] In one embodiment, the autonomous navigation processing unit (130) may generate source data for objects placed on the grid. The source data may include the position of the object, the size and type of the object, the identifier of the object, the velocity of the object, and the attitude of the object.

[0124] In one embodiment, the autonomous navigation processing unit (130) can generate dynamic graphic elements based on generated source data.

[0125] In one embodiment, the autonomous navigation processing unit (130) may generate dynamic graphic elements in different ways depending on the classification of the object. In one embodiment, the classification of the object may relate to whether the object is dynamic or static.

[0126] In one embodiment, generating a dynamic graphic element may include assigning a graphic asset. The graphic asset may be a pre-implemented shape model based on the type of object.

[0127] FIG. 6 is a diagram illustrating in detail a ship graphic interface according to another embodiment of the present disclosure.

[0128] Referring to FIG. 6, a ship graphic interface (600) is illustrated, and the ship graphic interface (600) may correspond to the ship graphic interface (410) described above with reference to FIG. 4.

[0129] However, the ship graphic interface (600) differs from the aforementioned ship graphic interface (500) in that it may include a dynamic graphic interface (610) configured to display dynamic graphic elements regarding the ship and objects surrounding the ship depending on the viewpoint from which the ship is viewed from the air.

[0130] In one embodiment, based on receiving input interacting with the point-of-view state object (506) described above with reference to FIG. 5, the autonomous navigation processing unit (130) or interface device may switch the dynamic graphical interface (510) according to a third-person viewpoint to a dynamic graphical interface (610) according to a viewpoint looking at the ship from the air.

[0131] Conversely, in one embodiment, based on receiving input interacting with a view state object (601), the autonomous navigation processing unit (130) or interface device may switch a dynamic graphical interface (610) according to a viewpoint looking at the ship from the air to a dynamic graphical interface (510) according to a third-person viewpoint.

[0132] Referring to FIG. 6, graphic elements relating to a vessel included in a dynamic graphic interface (610), graphic elements relating to a berth located around the vessel, and graphic elements relating to a berth location located around the vessel (620) are illustrated. The graphic elements included in the dynamic graphic interface (610) may dynamically change as the vessel moves or objects around the vessel move.

[0133] In one embodiment, a user of the vessel may interact with a graphical element (620) regarding a berthing location. In one embodiment, the user of the vessel may set a berthing location by interacting with the graphical element (620) regarding a berthing location while the graphical element (620) regarding a berthing location is displayed on the vessel graphical interface (600). In one embodiment, the autonomous navigation processing device (130) may set a berthing location based on receiving an input for interacting with the graphical element (620) regarding a berthing location, and may initiate automatic berthing based on the set berthing location.

[0134] In one embodiment, the dynamic graphical interface (610) may include graphical elements corresponding to a risk area. In the present disclosure, a risk area may refer to an area with a high risk level when it includes objects outside the vessel. In one embodiment, the risk area may be set as an area extending a preset distance from the vessel's own area.

[0135] Referring to FIG. 6, a graphic element (602) corresponding to a risk area included in a dynamic graphic interface (610) is illustrated.

[0136] Unlike the one illustrated in FIG. 6, in one embodiment, the dynamic graphical interface (610) may not include graphical elements corresponding to the risk area.

[0137] Meanwhile, although not illustrated in FIG. 6, in one embodiment, the dynamic graphical interface (610) may include graphic elements corresponding to a deformed risk area. In the present disclosure, the deformed risk area may be an area for intuitively providing information that an object surrounding the ship is close to the ship. In one embodiment, the device providing the interface or the autonomous navigation processing device (130) may determine the deformed risk area by deforming at least a portion of the risk area based on information about the surrounding objects and the risk area. For example, when an object approaches the risk area, a graphic element corresponding to the deformed risk area may be generated and displayed in a distorted form in the risk area close to the object.

[0138] Among the specific configurations of the ship graphic interface (600) of FIG. 6, the description regarding the same configuration as the ship graphic interface (500) of FIG. 5 can be equally applied to the description regarding the ship graphic interface (500) described above with reference to FIG. 5.

[0139] FIGS. 7A to 7E are diagrams illustrating a ship graphic interface during a process in which automatic docking is performed according to one embodiment of the present disclosure.

[0140] Figure 7a illustrates a ship graphic interface in the approaching phase. On the left side of Figure 7a, a ship graphic interface (711) in the approaching phase from a third-person perspective is illustrated, and on the right side of Figure 7a, a ship graphic interface (712) in the approaching phase from a perspective looking at the ship from the air is illustrated.

[0141] The approaching phase may refer to a stage during the berthing phases within the automatic berthing mode in which a vessel approaches a fixed position. In the present disclosure, the fixed position may refer to a position where the vessel's speed is reduced before entering the berthing position or berthing area, and where berthing is stably performed at a consistent position and attitude. In other words, the fixed position may be understood as the vessel's destination during the approaching phase and the starting point of the docking phase.

[0142] In one embodiment, upon receiving input interacting with a graphic element (520) regarding a docking location or a graphic element (620) regarding a docking location, a ship graphic interface (711) or a ship graphic interface (721) may be generated and displayed when an automatic docking mode is initiated.

[0143] In one embodiment, the vessel graphical interface in automatic berthing mode may include a berthing stage display object (701). The berthing stage display object (701) may be an object configured to display information about a berthing stage corresponding to the current vessel among the berthing stages in automatic berthing mode. In one embodiment, the device providing the interface or the autonomous navigation processing device (130) may receive or detect a berthing stage and generate a berthing stage display object based on the received or detected berthing stage.

[0144] Referring to FIG. 7A, it is illustrated that the ship graphic interface (711) includes a berthing phase indicator object (701) indicating that the berthing phase is an approaching phase, and the ship graphic interface (712) includes a berthing phase indicator object (702) indicating that the berthing phase is an approaching phase.

[0145] In one embodiment, the vessel graphical interface in automatic berthing mode may include graphical elements relating to berthing positions. Here, the berthing position may be a berthing position set by the user or automatically set.

[0146] Referring to FIG. 7A, it is illustrated that the ship graphic interface (711) includes a graphic element (703) relating to a berthing position, and the ship graphic interface (712) includes a graphic element (704) relating to a berthing position.

[0147] In one embodiment, the vessel graphical interface in automatic docking mode may include an automatic docking interrupt object. In one embodiment, a user of the vessel may interact with the automatic docking interrupt object. In one embodiment, the autonomous navigation processing unit (130) may interrupt automatic docking and switch the vessel's navigation mode to manual navigation mode based on receiving input interacting with the automatic docking interrupt object.

[0148] Referring to Fig. 7a, when the berthing phase is the approaching phase, the ship graphic interface (711) may simultaneously display the ship's berthing route (707) and future route (708). More specifically, when the ship's berthing phase is the approaching phase in which the ship approaches the berth, the ship graphic interface (711) may include and display the future route and berthing route as berthing phase display objects. At this time, the future route refers to the route along which the ship is expected to move at a future point in time based on the current position, and the berthing route refers to the route along which the ship is expected to move while berthing. Referring to Fig. 7a, it can be seen that the operation area (411) included in the ship graphic interface (410) of Fig. 4 is not displayed in the ship graphic interface (711) of Fig. 7a.

[0149] Referring to FIG. 7a, the ship graphic interface (711) is illustrated to include an automatic docking stop object (705), and the ship graphic interface (712) is illustrated to include an automatic docking stop object (706).

[0150] Fig. 7b illustrates a ship graphic interface in the position control phase (positioning phase). On the left side of Fig. 7b, a ship graphic interface (721) in the position control phase from a third-person perspective is illustrated, and on the right side of Fig. 7b, a ship graphic interface (722) in the position control phase from a viewpoint looking at the ship from the air is illustrated.

[0151] The position control phase may refer to a phase in which, among the berthing phases in the automatic berthing mode, the vessel is controlled to move to a fixed position or maintain a fixed position. Unlike vehicles on land, vessels are located at the sea surface, making it impossible to maintain a relative position with respect to the surrounding environment. Therefore, dynamic positioning may be required to control the position. From this perspective, the fixed position may be referred to as a DP (dynamic positioning) point, which is a reference point for dynamic positioning control. Meanwhile, when the vessel reaches a fixed position or a position close to the fixed position, the autonomous navigation processing device (130) may enter a position control phase in which dynamic position control is performed.

[0152] In one embodiment, the autonomous navigation processing unit (130) may calculate a fixed position based on entering the approach phase. In one embodiment, the autonomous navigation processing unit (130) may enter the position control phase based on the distance between the vessel's position and the fixed position. For example, the autonomous navigation processing unit (130) may enter the position control phase based on the distance between the vessel's position and the fixed position being less than or equal to a threshold value.

[0153] As described above, the vessel graphical interface in auto-berthing mode may include a berthing step indicator object, a graphical element relating to the berthing position, and / or an auto-berthing interrupt object.

[0154] Referring to Figure 7b, in the position control phase (positioning phase), the surrounding berth image may change according to changes in the vessel's heading. If the berthing phase is a positioning phase that determines the vessel's berthing position at the berth, the vessel graphic interface (721) may display the process of the surrounding environment, including the berth, rotating relative to the vessel.

[0155] For example, when a ship is parked forward, a berth image in which the berth is located in front of the ship cluster is displayed, and when a ship is parked backward, a berth image in which the berth is located behind the ship cluster can be displayed through the ship graphic interface (721) as shown in FIG. 7c.

[0156] Referring to FIG. 7b, it is illustrated that each of the ship graphic interface (721) and the ship graphic interface (722) includes a berthing step display object, a graphic element relating to a berthing position, and an automatic berthing stop object.

[0157] Fig. 7c illustrates a ship graphic interface in the docking phase (or, entering phase). On the left side of Fig. 7c, a ship graphic interface (731) in the docking phase from a third-person perspective is illustrated, and on the right side of Fig. 7c, a ship graphic interface (732) in the docking phase from a perspective looking at the ship from the air is illustrated.

[0158] The docking phase (or entering phase) may refer to the stage during which a vessel enters a docking position or docking area after reaching a fixed position within the automatic docking mode. The docking phase may be initiated only when the vessel is in a stable state. The autonomous navigation processing unit (130) can control the vessel during the docking phase by calculating throttle and steering control values ​​considering state errors.

[0159] As described above, the vessel graphical interface in auto-berthing mode may include a berthing step indicator object, a graphical element relating to the berthing position, and / or an auto-berthing interrupt object.

[0160] Referring to FIG. 7c, it is illustrated that each of the ship graphic interface (731) and the ship graphic interface (732) includes a berthing step display object, a graphic element relating to a berthing position, and an automatic berthing stop object.

[0161] Fig. 7d illustrates a ship graphic interface in the pushing phase. On the left side of Fig. 7d, a ship graphic interface (741) in the pushing phase from a third-person perspective is illustrated, and on the right side of Fig. 7d, a ship graphic interface (742) in the pushing phase from a perspective looking at the ship from the air is illustrated.

[0162] The pushing phase may refer to a stage within the automatic berthing mode where pushing control is performed. Pushing control may refer to a control that pushes a vessel closer to a berth in a preset direction from a berthing position or a position close to the berthing position. Here, the preset direction may be preset by matching data regarding the berth or docking position, or may be preset by user input.

[0163] In one embodiment, the autonomous navigation processing unit (130) may perform pushing control by performing only control regarding the heading direction, simultaneously performing control regarding the heading direction and control regarding the heading angle, or simultaneously performing control regarding the side direction and control regarding the heading angle of the vessel, based on the speed of the side direction of the vessel and one or more threshold values ​​when the vessel enters the pushing phase. Specifically, only control regarding the heading direction may be performed based on the speed of the side direction being less than a first threshold value, control regarding the side direction and control regarding the heading angle may be performed simultaneously based on the speed of the side direction being greater than a second threshold value, and control regarding the side direction and control regarding the heading angle may be performed simultaneously based on the speed of the side direction being greater than or equal to the first threshold value and less than or equal to the second threshold value. For example, the first threshold value may be -0.1 m / s, and the second threshold value may be 0.2 m / s.

[0164] As described above, the vessel graphical interface in auto-berthing mode may include a berthing step indicator object, a graphical element relating to the berthing position, and / or an auto-berthing interrupt object.

[0165] Referring to FIG. 7d, it is illustrated that each of the ship graphic interface (741) and the ship graphic interface (742) includes a berthing step display object, a graphic element relating to a berthing position, and an automatic berthing stop object.

[0166] In one embodiment, the ship graphical interface in the pushing phase may include an object indicating the direction of the pushing control.

[0167] Referring to FIG. 7d, it is illustrated that each of the ship graphic interface (741) and the ship graphic interface (742) includes an object indicating the direction of the pushing control.

[0168] Figure 7e illustrates a vessel graphic interface in a state where automatic docking is complete. On the left side of Figure 7e, a vessel graphic interface (751) in a state where automatic docking is complete from a third-person perspective is illustrated, and on the right side of Figure 7e, a vessel graphic interface (752) in a state where automatic docking is complete from a viewpoint looking at the vessel from the air is illustrated.

[0169] As described above, the vessel graphical interface in auto-berthing mode may include a berthing step indicator object, a graphical element relating to the berthing position, and / or an auto-berthing interrupt object.

[0170] Referring to FIG. 7e, it is illustrated that each of the ship graphic interface (751) and the ship graphic interface (752) includes a berthing step display object, a graphic element relating to a berthing position, and an automatic berthing stop object.

[0171] In one embodiment, a user of the vessel may interact with an automatic docking interrupt object. In one embodiment, the autonomous navigation processing device (130) may complete automatic docking based on receiving input interacting with the automatic docking interrupt object.

[0172] Meanwhile, although not illustrated in FIG. 4 or FIGS. 7A-7E, in one embodiment, the main interface or ship graphical interface may include a warning message in automatic docking mode. Here, the warning message may refer to a message that guides or warns the user of the ship to visually inspect objects around the ship or to check image data collected by the camera. Through the warning message, the user of the ship may be encouraged to pay close attention to the ship's surroundings or to check images collected by the camera (e.g., through the surround view interface or individual camera view interface described below), thereby enhancing safety. The user may interact with the displayed warning message to deactivate the warning message.

[0173] Figure 7f illustrates an embodiment of a vessel graphical interface in a state where automatic docking is completed.

[0174] On the left side of FIG. 7f, a ship graphic interface (761) is shown in a state in which automatic docking is completed according to a third-person viewpoint, and on the right side of FIG. 7f, a ship graphic interface (762) is shown in a state in which automatic docking is completed according to a viewpoint looking at the ship from the air.

[0175] Referring to FIG. 7f, after automatic docking is completed and before the Auto Mode is released, an anchoring function may be automatically performed. At this time, a concentric circle-shaped anchoring indicator object (763) may be additionally displayed below the ship cluster to visually indicate that the anchoring function is being performed. When the ship's engine is turned off, the anchoring indicator object (763) may disappear, and the display may change as shown in FIG. 7f.

[0176] FIGS. 8A and 8B are diagrams illustrating a customizable interface according to one embodiment of the present disclosure.

[0177] In one embodiment, the interface, specifically the main interface, may include a resizing object. Referring to FIG. 4 described above, the interface (400) is illustrated as including a resizing object positioned between the ship graphic interface (410) and the map interface (420).

[0178] In one embodiment, a user of the vessel may interact with a resizing object. For example, the user of the vessel may drag the resizing object to resize components within the interface. In one embodiment, the device providing the interface or the autonomous navigation processing unit (130) may change the configuration of the interface based on receiving input interacting with the resizing object.

[0179] FIG. 8a illustrates an interface in which the area occupied by the ship graphic interface becomes smaller and the area occupied by the map interface becomes larger as the size adjustment object (801) is moved to the left side of the interface. FIG. 8b illustrates an interface in which the area occupied by the ship graphic interface becomes larger and the area occupied by the map interface becomes smaller as the size adjustment object (801) is moved to the right side of the interface.

[0180] For example, a user may customize the map interface to be displayed in a larger size by interacting with the scale object (801) to facilitate planning a route for the vessel's navigation or navigating the location of a berth. For example, a user may customize the ship graphical interface to be displayed in a larger size by interacting with the scale object (801) to facilitate identifying objects around the vessel or checking status information about the vessel.

[0181] FIG. 9 is a diagram illustrating a customizable interface according to another embodiment of the present disclosure.

[0182] In one embodiment, the configuration of the interface, specifically the sub-interfaces included in the main interface, may be customized. The interface (400) described above with reference to FIG. 4 may include a ship graphic interface (410) and a map interface (420), but other sub-interfaces may be created and displayed in place of the ship graphic interface (410) or the map interface (420).

[0183] In one embodiment, the sub-interface may be a surround view interface. That is, the interface may include a ship graphics interface, a map interface, and / or a surround view interface. The surround view interface may be an interface configured to display a surround view generated based on image data collected by a camera mounted on the ship.

[0184] Referring to FIG. 9, an interface (900) is illustrated that includes a ship graphic interface (910) and a surround view interface (920). The interface (900) illustrated in FIG. 9 may be an interface that is created and displayed by a user of the ship customizing the surround view interface (920) to be displayed instead of the map interface, or by switching the map interface to the surround view interface (920).

[0185] In one embodiment, a user of the vessel may interact with a surround view interface (920). In one embodiment, the user of the vessel may change the viewpoint or zoom in on a specific area by interacting with the surround view interface (920). In one embodiment, a device providing an interface or an autonomous navigation processing device (130) may generate a surround view interface (920) corresponding to the input based on receiving an input interacting with the surround view interface (920).

[0186] In FIG. 9, the interface (900) is shown as including a ship graphic interface (910) and a surround view interface (920), but depending on the user's settings, the interface (900) may also include a surround view interface (920) and a map interface.

[0187] For example, a user can customize the interface to display a surround view interface, making it easier to observe and assess the surroundings of an actual vessel. In one embodiment, the user can configure the interface according to the operating mode of each vessel.

[0188] Meanwhile, in another embodiment, the device providing the interface or the autonomous navigation processing device (130) can configure the main interface based on the navigation mode of the ship.

[0189] For example, the device providing the interface or the autonomous navigation processing device (130) may configure the main interface as a ship graphic interface (910) and a surround view interface (920) based on whether the ship's navigation mode is one of the automatic navigation modes. Conversely, for example, the device providing the interface or the autonomous navigation processing device (130) may configure the main interface as a ship graphic interface (910) and a map interface based on whether the ship's navigation mode is a manual navigation mode.

[0190] For another example, the device providing the interface or the autonomous navigation processing device (130) may configure the main interface as a ship graphic interface (910) and a surround view interface (920) based on whether the ship's navigation mode is an automatic berthing mode. Conversely, for example, the device providing the interface or the autonomous navigation processing device (130) may configure the main interface as a ship graphic interface (910) and a map interface based on whether the ship's navigation mode is not an automatic berthing mode.

[0191] In one embodiment, the surround view interface (920) may include an individual camera view controller (901). The individual camera view controller (901) may be an object configured to individually view image data collected by each of one or more cameras mounted on the vessel. As illustrated in FIG. 9, the individual camera view controller (901) may include a first component that receives an input from a user to select at least one of one or more cameras mounted on the vessel, and a second component that displays an image captured from the camera selected by the user.

[0192] As illustrated in Fig. 9, the first configuration can be designed considering the spacing and viewing angles of cameras installed on an actual ship. For example, if six cameras are installed on a ship, an octagon can be divided into three parts in the bow, side, and stern directions of the ship according to the installation spacing of the cameras, the bow direction area can be divided into three parts according to the viewing angles of the corresponding cameras, and the side direction area can be divided into two parts according to the viewing angles of the corresponding cameras. According to an embodiment, a ship-shaped figure can be placed in the center of the outline of the octagon. Accordingly, it can be intuitively indicated that each of the divided areas of the first configuration corresponds to the cameras in the bow, side, and stern directions. According to an embodiment, the outline of the octagon of the first configuration can be implemented as a circle or a square.

[0193] The second configuration may be configured to display an image corresponding to a camera selected by the user through the first configuration. That is, when the user interacts with the first configuration to select at least one of a plurality of cameras installed on the ship, the second configuration may display an image corresponding to the selected camera, thereby improving user intuitiveness. In addition, the second configuration may be configured to display the selected camera and its corresponding image together by changing the color or brightness of the segmented area of ​​the first configuration selected by the user.

[0194] Referring to FIG. 9, an individual camera view controller (901) is illustrated as including a camera selection interface configured to select one of one or more cameras mounted on the vessel, and an individual camera view interface configured to display image data collected by the selected camera. As illustrated in FIG. 9, the camera selection interface may be configured to indicate to the user which camera has been selected.

[0195] In one embodiment, a user can interact with an individual camera view controller (901).

[0196] In one embodiment, a user may select a camera by interacting with an individual camera view controller (901). In one embodiment, the device providing the interface or the autonomous navigation processing unit (130) may generate and display a camera selection interface to identify the selected camera based on receiving input interacting with the individual camera view controller (901), and may generate and display an individual camera view interface based on image data collected by the selected camera. For example, the user interaction may be a press input on the camera selection interface.

[0197] In one embodiment, a user may interact with an individual camera view controller (901) to cause image data collected by a selected camera to be enlarged and displayed. In one embodiment, a device providing an interface or an autonomous navigation processing device (130) may cause image data collected by a selected camera to be displayed on a surround view interface (920) based on receiving an input for interacting with the individual camera view controller (901). For example, the user interaction may be a press input on the individual camera view interface.

[0198] In one embodiment, a user may interact with an individual camera view controller (901) to change the position of the individual camera view controller (901) within a surround view interface (920). In one embodiment, a device providing an interface or an autonomous navigation processing device (130) may generate and display a surround view interface (920) to move the individual camera view controller (901) based on receiving input interacting with the individual camera view controller (901). For example, the user interaction may be a long press and drag input to the individual camera view controller (901).

[0199] As described above, a user may interact with the individual camera view controller (901) to select a camera, but in one embodiment, the user may also interact with the individual camera view controller (901) to set the selected camera to be automatically switched. In one embodiment, the device providing the interface or the autonomous navigation processing unit (130) may generate and display a camera selection interface and an individual camera view interface to automatically switch the selected camera based on receiving input interacting with the individual camera view controller (901). For example, the individual camera view controller (901) may include an auto / manual switch object, and the user interaction may be a press input on the auto / manual switch object.

[0200] In one embodiment, if the selected cameras are set to switch automatically, the selected cameras may switch in a preset order at preset intervals. For example, the preset interval may be 2 seconds.

[0201] In one embodiment, the selected camera is set to automatically switch, and when an object approaches the vessel, the selected camera may switch based on the location of the object. For example, if an object is detected that is less than a threshold distance from the vessel's bow, the selected camera may switch to a camera that collects image data in the bow direction of the vessel.

[0202] FIG. 10 is a flowchart of a method according to one embodiment of the present disclosure.

[0203] The method illustrated in FIG. 10 may be a method for providing the aforementioned interface. The method illustrated in FIG. 10 may be performed by a device or autonomous operation processing device (130) providing the aforementioned interface, specifically, a processor included in the device or autonomous operation processing device (130) providing the interface.

[0204] In one embodiment, at step 1010, the processor may receive data collected by sensors mounted on the vessel and data regarding the vessel's operating mode.

[0205] In one embodiment, at step 1020, the processor may generate an interface based on data collected by the sensors and data regarding the operating mode.

[0206] In one embodiment, the interface includes dynamic graphical elements relating to the vessel and objects around the vessel, which can change based on the operating mode.

[0207] In one embodiment, the interface may include a ship graphic interface configured to display dynamic graphic elements and a map interface configured to display a map of the ship's location.

[0208] In one embodiment, the map interface may switch to a surround view interface based on whether the navigation mode is one of the automatic navigation modes.

[0209] In one embodiment, the surround view interface may include individual camera view controllers configured to individually view image data collected by each of one or more cameras mounted on the vessel.

[0210] In one embodiment, the vessel graphical interface may include a speed information object, a sailing mode information object, a fuel information object, or a depth information object.

[0211] In one embodiment, the vessel graphical interface includes a heading angle maintenance mode initiation object, a position maintenance mode initiation object, or a path tracking mode initiation object, and based on receiving input for interacting with the heading angle maintenance mode initiation object, the navigation mode can be switched to the heading angle maintenance mode, based on receiving input for interacting with the position maintenance mode initiation object, the navigation mode can be switched to the position maintenance mode, and based on receiving input for interacting with the path tracking mode initiation object, the navigation mode can be switched to the path tracking mode.

[0212] In one embodiment, the ship graphical interface may be generated from either a third-person perspective or an aerial perspective of the ship.

[0213] In one embodiment, the ship graphical interface includes a viewpoint state object, and the processor can further perform the step of switching the ship graphical interface from a third-person viewpoint to an aerial viewpoint of the ship, or from an aerial viewpoint of the ship to a third-person viewpoint, based on receiving input interacting with the viewpoint state object.

[0214] In one embodiment, the ship graphical interface may include graphical elements corresponding to a risk area, which is an area of ​​high risk when extra-vessel objects are included.

[0215] In one embodiment, the ship graphical interface may include a resizable object.

[0216] In one embodiment, the processor may further perform the step of changing the dimensions of the ship graphical interface and the map interface based on receiving input interacting with the resizing object.

[0217] In one embodiment, the processor may further perform the steps of: setting a berthing position corresponding to the input based on receiving an input interacting with the graphic element relating to the berthing position while the graphic element relating to the berthing position is displayed on the ship graphic interface; and initiating automatic berthing based on the set berthing position.

[0218] In one embodiment, the vessel graphical interface includes a berthing stage display object, and the processor may further perform the step of detecting a berthing stage corresponding to the current vessel among berthing stages in the automatic berthing mode based on initiating automatic berthing, and generating a berthing stage display object based on the berthing stage.

[0219] In one embodiment, the processor may further perform the step of generating a dynamic graphics element.

[0220] In one embodiment, generating the dynamic graphic elements may include generating a grid centered around the ship and placing objects on the grid based on information about objects surrounding the ship.

[0221] FIG. 11 is a block diagram of a device according to one embodiment of the present disclosure.

[0222] The device (1100) illustrated in FIG. 11 may be a device providing the aforementioned interface or an autonomous operation processing device (130).

[0223] Referring to FIG. 11, the device (1100) may include a communication unit (1110), a processor (1120), and a database (1130). Only components related to the embodiment are illustrated in the device (1100) of FIG. 11. Therefore, those skilled in the art will understand that other general components may be included in addition to the components illustrated in FIG. 11.

[0224] The communication unit (1110) may include one or more components that enable wired / wireless communication with an external server or external device. For example, the communication unit (1110) may include at least one of a short-range communication unit (not shown), a mobile communication unit (not shown), and a broadcast receiving unit (not shown).

[0225] DB (1130) is hardware that stores various data processed within the device (1100) and can store programs for processing and controlling the processor (1120). DB (1130) can store payment information, user information, etc.

[0226] DB (1130) may include random access memory (RAM) such as dynamic random access memory (DRAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), CD-ROM, Blu-ray or other optical disk storage, hard disk drive (HDD), solid state drive (SSD), or flash memory.

[0227] The processor (1120) controls the overall operation of the device (1100). For example, the processor (1120) can control the input unit (not shown), the display (not shown), the communication unit (1110), the DB (1130), etc., by executing programs stored in the DB (1130). The processor (1120) can control the operation of the device (1100) by executing programs stored in the DB (1130).

[0228] The processor (1120) can control at least some of the operations of the device (1100) described above in FIGS. 1 to 10.

[0229] The processor (1120) may be implemented using at least one of application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, micro-controllers, microprocessors, and other electrical units for performing functions.

[0230] In one embodiment, the device (1100) may be a mobile electronic device. For example, the device (1100) may be implemented as a smartphone, tablet PC, PC, smart TV, personal digital assistant (PDA), laptop, media player, navigation device, camera-equipped device, or other mobile electronic device. Furthermore, the device (1100) may be implemented as a wearable device, such as a watch, glasses, hair band, or ring, equipped with communication and data processing capabilities.

[0231] Embodiments according to the present invention may be implemented in the form of a computer program that can be executed through various components on a computer, and such a computer program may be recorded on a computer-readable medium. In this case, the medium may include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical recording media such as CD-ROMs and DVDs, magneto-optical media such as floptical disks, and hardware devices specifically configured to store and execute program instructions, such as ROMs, RAMs, and flash memories.

[0232] Meanwhile, the computer program may be specifically designed and constructed for the present invention, or may be one known and available to those skilled in the computer software field. Examples of computer programs may include not only machine language code, such as that generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like.

[0233] According to one embodiment, the method according to various embodiments of the present disclosure may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) through an application store (e.g., Play Store™) or directly between two user devices. In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0234] Unless the steps constituting the method according to the present invention are explicitly described in a specific order or are otherwise described in a different order, the steps may be performed in any appropriate order. The present invention is not necessarily limited to the order in which the steps are described. The use of all examples or exemplary terms (e.g., “for example,” “etc.”) in the present invention is merely intended to illustrate the present invention in more detail, and the scope of the present invention is not limited by the examples or exemplary terms unless otherwise defined by the claims. Furthermore, those skilled in the art will appreciate that various modifications, combinations, and variations can be configured according to design conditions and factors within the scope of the appended claims or their equivalents.

[0235] Therefore, the idea of ​​the present invention should not be limited to the embodiments described above, and not only the scope of the patent claims described below but also all scopes equivalent to or equivalently modified from the scope of the patent claims are considered to fall within the scope of the idea of ​​the present invention.

Claims

1. A method for providing a ship operation information interface, A step of receiving data collected by sensors mounted on a ship and data regarding the ship's operating mode; and A step of generating an interface based on data collected by the sensor and data regarding the operating mode; Including, but not limited to, The above interface is, A ship graphic interface configured to display dynamic graphic elements relating to the ship and objects surrounding the ship; and A map interface configured to display a map of the location of the vessel; Including, Based on the above operation mode, it changes, method.

2. In paragraph 1, The above ship graphic interface is, A driving area indicating an area in which the vessel can be driven in auto mode; The planned route on which the said vessel is to move; and A method of displaying a future route, which is a route that the vessel is expected to travel at a future point in time.

3. In paragraph 2, Based on whether the above operation mode is one of the automatic operation modes, the map interface is switched to a surround view interface. method.

4. In paragraph 3, The above surround view interface is, Including an individual camera view controller configured to individually check image data collected by each of one or more cameras mounted on the vessel, method.

5. In paragraph 4, The above individual camera view controllers are: Including a shape designed considering the spacing and viewing angle of the camera installed on the above ship, method.

6. In paragraph 5, The above individual camera view controllers are: Through the above figure, the camera screen corresponding to the above figure is further displayed as a sub-screen. method.

7. In paragraph 2, The above ship graphic interface is, Containing a speed information object, a navigation mode information object, a fuel information object or a depth information object, method.

8. In paragraph 2, The above ship graphic interface is, Contains a player angle maintenance mode initiation object, a position maintenance mode initiation object, or a path tracking mode initiation object, Based on receiving an input interacting with the above player angle maintenance mode initiation object, the operation mode is switched to the player angle maintenance mode, Based on receiving an input interacting with the position holding mode initiating object, the operating mode is switched to the position holding mode, Based on receiving an input interacting with the above path tracking mode initiation object, the operating mode is switched to the path tracking mode. method.

9. In paragraph 2, A step of setting a berthing position corresponding to an input based on receiving an input interacting with a graphic element regarding the berthing position while a graphic element regarding the berthing position is displayed on the ship graphic interface; and A step of initiating automatic docking based on the above-set docking position; including, method.

10. In paragraph 7, The above ship graphic interface is, Contains a display object for the eyepiece stage, The above method, A step of detecting a berthing stage corresponding to the current vessel among berthing stages in the automatic berthing mode based on initiating the automatic berthing; and A step of generating the eyepiece stage display object based on the eyepiece stage; including more, method.

11. In paragraph 10, If the above berthing stage is an approaching stage in which the vessel approaches the berth, The above ship graphic interface displays future routes and berthing routes as berthing stage display objects, The above future route is the route along which the vessel is expected to travel in the future, The above berthing route is a route along which the vessel is expected to move while berthing.

12. In paragraph 10, If the above docking step is a positioning step that determines the docking position on the berth, A method in which the ship graphic interface displays a process in which the surrounding environment including the berth rotates relative to the ship.

13. In paragraph 10, If the above berthing stage is the pushing stage in which the vessel enters the berth, The above ship graphic interface includes an object that displays the direction of the pushing control as the berthing stage display object. method.

14. In paragraph 10, If the above berthing stage is a completion stage in which the berthing of the vessel is completed, The above ship graphic interface includes an anchoring indicator object in the shape of concentric circles that displays the location where the ship has completed docking. method.

15. In paragraph 2, The above ship graphic interface is, It contains a viewpoint state object that is generated based on either a third-person view or a viewpoint looking at the ship from the air, The above method, A step of switching the ship graphical interface from the third-person viewpoint to a viewpoint looking at the ship from the air, or from a viewpoint looking at the ship from the air to the third-person viewpoint, based on receiving an input interacting with the viewpoint state object; more inclusive, method.

16. In paragraph 2, The above ship graphic interface is, Contains a resizing object, A step of changing the area of ​​the ship graphic interface and the map interface based on receiving an input interacting with the size adjustment object; including more, method.

17. A device providing a ship operation information interface, memory in which at least one program is stored; and A processor that operates by executing at least one program; Including, but not limited to, The above processor, Receive data collected by sensors mounted on the ship and data regarding the ship's operating mode, Based on the data collected by the above sensor and the data regarding the above operating mode, an interface is generated, The above interface is, A ship graphic interface configured to display three-dimensional dynamic graphic elements relating to the ship and objects surrounding the ship; A map interface configured to display a map of the location of the vessel; and At least a portion of a surround view interface including an individual camera view controller configured to individually view image data collected by each of one or more cameras mounted on the vessel, Based on the above operation mode, it changes, device.

18. A computer-readable recording medium recording a program for executing the method according to paragraph 1 on a computer.

Images