Navigation assistance system, navigation assistance method, and program

Abstract

[Program] To provide a navigation assistance system capable of orienting an imaging direction of a camera to a direction of a target present around a vessel. [Solution] This navigation assistance system comprises: a target sensor that detects the position of a target present around a ship; an azimuth sensor that detects the bow direction of the vessel; a camera that is installed on the vessel and has a posture drive unit that changes the imaging direction; an imaging control unit that controls the posture drive unit to orient the imaging direction of the camera to the direction of the target with reference to the vessel; and a display control unit that displays an image generated by the camera.

Description

Navigation Support System, Navigation Support Method, and Program

[0001] The present invention relates to a navigation support system, a navigation support method, and a program.

[0002] Conventionally, a technique for detecting a maritime target such as a ship in an image captured by a camera mounted on a ship by image recognition is known.

[0003] International Publication No. 2020 / 049634

[0004] By the way, there is a type of camera that can change the imaging direction, and it is convenient if the imaging direction of the camera can be directed toward the direction of the target existing around the ship.

[0005] The present invention has been made in view of the above problems, and its main object is to provide a navigation support system, a navigation support method, and a program capable of directing the imaging direction of a camera in the direction of a target existing around a ship.

[0006] To solve the above problems, a navigation support system according to one aspect of the present invention includes a target sensor that detects the position of a target existing around a ship, a direction sensor that detects the bow direction of the ship, a camera installed on the ship and having a posture drive unit that changes the imaging direction, an imaging control unit that controls the posture drive unit to direct the imaging direction of the camera in the direction of the target with respect to the ship, and a display control unit that displays an image generated by the camera. According to this, it becomes possible to direct the imaging direction of the camera in the direction of a target existing around the ship.

[0007] In the above aspect, the camera may further include a lens drive unit that changes the focal length of a zoom lens, and the imaging control unit may control the lens drive unit to zoom in on the target. According to this, it also becomes possible to zoom in on the target.

[0008] In the above embodiment, the target sensor is a radar or an AIS (Automatic Identification System), and the imaging control unit may accept a target designation by the user and direct the imaging direction of the camera toward the designated target. This makes it possible to direct the imaging direction of the camera toward the target designated by the user.

[0009] In the above embodiment, the target sensor includes an image recognition unit that recognizes the region of the target in the image, and the imaging control unit may control the lens drive unit so that the region of the target is included in the image within a predetermined size. This makes it possible to zoom in on the target so that the region of the target is included within a predetermined size.

[0010] In the above embodiment, the target sensor includes another camera installed on the vessel and an image recognition unit that recognizes the region of the target in an image generated by the other camera. The imaging control unit may further include a calculation unit that moves the imaging direction of the camera along a straight line from the other camera toward the target, and calculates the distance from the vessel to the target based on the direction from the camera toward the target and the direction from the other camera toward the target when the region of the target is recognized in an image generated by the camera. This makes it possible to calculate the distance from the vessel to the target.

[0011] In the above embodiment, the target sensor may be a radar and further include an image recognition unit that recognizes the target in the image generated by the camera, and a determination unit that determines whether the target detected by the radar is a false image when the target is recognized in the image. This makes it possible to determine whether or not the target detected by the radar is a false image.

[0012] In the above embodiment, the target sensor includes a wide-angle camera with a wider angle than the camera, and an image recognition unit that recognizes the region of the target in the image generated by the wide-angle camera, and the imaging control unit may direct the imaging direction of the camera towards the direction of the target corresponding to the position of the region of the target in the image. This makes it possible to direct the imaging direction of the camera towards the direction of the target recognized in the image.

[0013] In the above embodiment, the target sensor is a radar that tracks the detected target, and the imaging control unit may sequentially update the imaging direction of the camera based on the position of the target sequentially output by the radar. This makes it possible to keep the imaging direction of the camera pointed towards the target.

[0014] In the above embodiment, the image recognition unit that recognizes the region of the target in the image generated by the camera and the display control unit may adjust the range of the image displayed on the screen so that the region of the target is maintained in a predetermined position on the screen. This makes it possible to maintain the target in a predetermined position on the screen even when the ship is rocking.

[0015] Furthermore, in another aspect of the present invention, a navigation support method involves detecting the position of targets present around the vessel using a target sensor, detecting the bow direction of the vessel using a compass sensor, and controlling an attitude drive unit that changes the imaging direction of a camera installed on the vessel to orient the camera's imaging direction toward the target relative to the vessel, and displaying the image generated by the camera. This makes it possible to orient the camera's imaging direction toward the target present around the vessel.

[0016] Furthermore, a program in another aspect of the present invention causes a computer to perform the following actions: acquire the position of targets present around the vessel as detected by a target sensor; acquire the bow direction of the vessel as detected by a compass sensor; control the attitude drive unit of a camera installed on the vessel to change the imaging direction horizontally, thereby orienting the camera's imaging direction toward the target relative to the vessel; and display the image generated by the camera. This makes it possible to orient the camera's imaging direction toward the target present around the vessel.

[0017] According to the present invention, it is possible to orient the camera's imaging direction toward the direction of a target present around the vessel.

[0018] This is a diagram showing an example of the system. This is a diagram showing an example of the camera. This is a diagram showing an example of the system. This is a diagram showing an example of the first designation process. This is a diagram showing an example of the first designation process. This is a diagram showing an example of the zoom process. This is a diagram showing an example of the zoom process. This is a diagram showing an example of the zoom process. This is a diagram showing an example of the zoom process. This is a diagram showing an example of the target tracking process. This is a diagram showing an example of the target tracking process. This is a diagram showing an example of the second designation process. This is a diagram showing an example of the second designation process. This is a diagram showing an example of the distance estimation process. This is a diagram showing an example of the distance estimation process. This is a diagram showing an example of the distance estimation process.

[0019] Embodiments of the present invention will be described below with reference to the drawings. In this specification and in each drawing, elements similar to those described above in relation to previously shown drawings will be denoted by the same reference numerals, and detailed descriptions may be omitted as appropriate.

[0020] Figure 1 is a block diagram showing an example configuration of the navigation support system 100. The navigation support system 100 is a system installed on a ship. In the following description, a ship on which the navigation support system 100 is installed will be referred to as "our ship," and other ships will be referred to as "other ships."

[0021] The navigation support system 100 comprises a control unit 1, a display unit 2, a radar 3, an AIS 4, a camera 5, a GNSS receiver 6, a direction sensor 7, an ECDIS 8, a wireless communication unit 9, and a ship handling control unit 10. These devices are connected to a network N, such as a LAN, and are capable of network communication with each other.

[0022] The control unit 1 includes a computer comprising a CPU, RAM, ROM, non-volatile memory, and an input / output interface. The CPU of the control unit 1 performs information processing according to a program loaded into RAM from ROM or non-volatile memory.

[0023] The program may be supplied via an information storage medium such as an optical disc or memory card, or via a communication network such as the Internet or a LAN.

[0024] The display unit 2 displays the display image generated by the control unit 1. The display unit 2 also displays radar images, camera images, or electronic charts.

[0025] The display unit 2 is, for example, a display device with a touch sensor, a so-called touch panel, which detects the position indicated on the screen by the user's finger or the like. However, it is not limited to this, and the indicated position may also be input by a pointing device such as a trackball.

[0026] Radar 3 emits radio waves around the vessel and receives the reflected waves, generating echo data based on the received signals. Radar 3 also identifies targets from the echo data and generates TT data (Target Tracking Data) representing the target's position and speed.

[0027] The AIS (Automatic Identification System) 4 receives AIS data from other vessels or shore-based control systems in the vicinity of the vessel. While AIS is not the only option, VDES (VHF Data Exchange System) may also be used. AIS data includes the identification code, name, position, course, speed, type, length, and destination of other vessels.

[0028] Camera 5 is a digital camera that captures images of the outside from the ship and generates image data. Camera 5 is installed, for example, on the ship's bridge, facing the bow. Camera 5 is a so-called PTZ camera, which has, for example, pan-tilt and optical zoom functions.

[0029] Camera 5 may include an image recognition unit that estimates the position and type of objects at sea, such as other vessels, contained in the captured image using an object detection model. The image recognition unit is not limited to camera 5, but may be implemented in other devices such as control unit 1.

[0030] The GNSS receiver 6 detects the ship's position based on radio waves received from the GNSS (Global Navigation Satellite System). The heading sensor 7 is, for example, a GPS compass or a gyrocompass, and detects the ship's bow direction.

[0031] The ECDIS (Electronic Chart Display and Information System) 8 obtains the ship's position from the GNSS receiver 6 and displays the ship's position on the electronic chart. The ECDIS 9 also displays the ship's planned route on the electronic chart. A GNSS plotter may be used instead of the ECDIS.

[0032] The wireless communication unit 9 includes wireless equipment for realizing satellite communications. The wireless communication unit 9 also includes wireless equipment for realizing ship-to-shore or ship-to-ship wireless communications using, for example, ultra-high frequency, very high frequency, short frequency, or medium-high frequency.

[0033] The ship steering control unit 10 is a control device for achieving autonomous navigation and controls the ship's steering gear. The ship steering control unit 10 may also control the ship's engine.

[0034] In this embodiment, the control unit 1 and the display unit 2 are independent devices, but the control unit 1 and the display unit 2 may be an integrated device.

[0035] Furthermore, the control unit 1 is not limited to an independent device, but may be integrated with other devices such as the ECDIS 8. In other words, some or all of the functions of the control unit 1 may be implemented by other devices.

[0036] Furthermore, the display unit 2 is not limited to an independent device; the display unit of another device, such as the ECDIS 8, may be used as the display unit 2 that displays the display image generated by the control unit 1.

[0037] Furthermore, the control unit 1 and the display unit 2 may be installed, for example, at a land-based control center and used to monitor vessels navigating within a controlled area.

[0038] Figure 2 is a block diagram showing an example configuration of camera 5. Camera 5 is a PTZ camera having pan-tilt and optical zoom functions, and comprises an imaging control unit 51, a signal processing unit 52, a pan drive unit 53, a tilt drive unit 54, and a lens drive unit 55.

[0039] The pan drive unit 53 changes the imaging direction of the camera 5 horizontally, and the tilt drive unit 54 changes the imaging direction of the camera 5 vertically. The lens drive unit 55 changes the focal length of the zoom lens 50 to zoom in or out on the subject.

[0040] The pan drive unit 53 and the tilt drive unit 54 are examples of attitude control units. Note that the control of the pan drive unit 53, the tilt drive unit 54, or the lens drive unit 55 is not limited to the imaging control unit 51, but may also be performed by the control unit 1.

[0041] Figure 3 is a block diagram showing an example of the functional configuration of the navigation support system 100. The control unit 1 of the navigation support system 100 includes image recognition units 11 and 12, an operation reception unit 13, an imaging command unit 14, and display control units 15 and 16. These functional units are realized by the CPU of the control unit 1 executing information processing according to a program.

[0042] As shown in the figure, the navigation support system 100 may include multiple cameras 5A, 5B and multiple display units 2A, 2B. Camera 5 is a collective term for cameras 5A, 5B, and display unit 2 is a collective term for display units 2A, 2B. Camera 5B, radar 3, and AIS 4 are examples of target sensors TS that detect the position of a target.

[0043] The image recognition units 11 and 12 acquire the images generated by the cameras 5A and 5B, and input the acquired images into the region recognition model to recognize the regions of the targets in the images. The region recognition model is a trained model by machine learning. Not limited to this, the region recognition model may be a rule-based calculation model.

[0044] The region recognition model is, for example, an object detection model such as YOLO (You Only Look Once) or SSD (Single Shot MultiBox Detector), and outputs a bounding box surrounding the objects included in the image. Not limited to this, the region recognition model may be a region segmentation model such as Semantic Segmentation or Instance Segmentation.

[0045] The operation reception unit 13 receives the operation input by the user detected by the operation unit 29. The operation unit 29 may be a touch sensor provided on the display unit 2B, or may be a pointing device such as a trackball.

[0046] The imaging command unit 14 designates the imaging direction of the camera 5A based on the operation input by the user, the detection result of the radar 3, or the recognition result of the image, etc. That is, the imaging command unit 14 designates the direction of the target with respect to the own ship as the imaging direction of the camera 5A.

[0047] The imaging control unit 51 of the camera 5A controls the pan drive unit 53 and the tilt drive unit 54 to direct the imaging direction of the camera 5 in the direction of the designated target. Further, the imaging control unit 51 of the camera 5A controls the lens drive unit 55 to zoom in on the designated target.

[0048] The display control units 15 and 16 display the images generated by the cameras 5A and 5B on the display units 2A and 2B together with the recognition results of the images, etc. Further, the display control unit 16 displays the symbols of the targets detected by the radar 3 or the AIS 4 on the display unit 2B so as to represent the positional relationship with the own ship.

[0049] In this embodiment, the display control units 15 and 16 display images on the two display units 2A and 2B respectively, but the system is not limited to this, and images may be displayed in two separate areas within the screen of a single display unit.

[0050] The specific processes of the navigation support methods implemented in the navigation support system 100 will be described below. The control unit 1 executes each process shown in the flowchart described below according to the program.

[0051] [First Designation Process] Figure 4 is a flowchart showing an example of the procedure for the first designation process. Figure 5 is a diagram for explaining the first designation process.

[0052] First, the control unit 1 acquires target data from the radar 3 or AIS 4 (S11). The target data is either TT data or AIS data and includes the positions of targets TG present around the ship.

[0053] Next, the control unit 1 displays the symbol of the target TG on the display unit 2B based on the acquired target data (S12, processing as the display control unit 16). The display unit 2B displays the symbol of the target TG as a PPI (plan position indicator).

[0054] Next, the control unit 1 receives the designation of the target TG by the user UZ (S13, processing as the operation reception unit 13). Specifically, the control unit 1 determines whether the symbol of the target TG displayed on the display unit 2B was indicated by the operation input of the user UZ.

[0055] When the target TG is specified by the user UZ (S13: YES), the control unit 1 specifies the direction of the target TG to the camera 5A (S14, processing as the imaging command unit 14), and causes the camera 5 to adjust the imaging direction (S15).

[0056] In the case of radar 3, the direction of the target TG is determined based on the ship's bow direction detected by the direction sensor 7. In the case of AIS 4, the direction of the target TG is calculated based on the target's position and the ship's bow direction detected by the direction sensor 7.

[0057] When the direction of the target TG is specified, the imaging control unit 51 of the camera 5 controls the pan drive unit 53 and the tilt drive unit 54 to adjust the imaging direction of the camera 5 so that it faces the specified direction of the target TG.

[0058] Furthermore, the control unit 1 performs a zoom process to zoom in on the target TG on the camera 5 (S16). The imaging control unit 51 of the camera 5 controls the lens drive unit 55 to zoom in on the target TG. The zoom process S16 will be described below.

[0059] [Zoom Processing] Figure 6 is a flowchart showing a specific example of the zoom processing S16 procedure. Figures 7 to 9 are diagrams for explaining the zoom processing S16.

[0060] First, the control unit 1 recognizes the regions BB1 and BB2 of targets TG1 and TG2 in the image GZ generated by the camera 5A (S61, processing as the image recognition unit 11).

[0061] As shown in Figure 7, the field of view FA of camera 5A and the resulting image GZ include targets TG1 and TG2, and the region recognition model recognizes regions BB1 and BB2 of targets TG1 and TG2. Regions BB1 and BB2 are boundary boxes surrounding targets TG1 and TG2. Of these, target TG1, located in the center of image GZ, is designated as the target to be imaged, and the imaging direction of camera 5A is directed toward target TG1.

[0062] Next, the control unit 1 zooms in on the target TG1 with the camera 5A so that the area BB1 of the target TG1 becomes a predetermined size (S62).

[0063] As shown in Figure 8, zooming in narrows the field of view FA of the camera 5A, and the target TG1 included in the image GZ is enlarged. For example, the control unit 1 zooms in until the gap CL between the periphery of the image GZ and the region BB1 of the target TG1 is a predetermined size. In the illustrated example, the gap CL in the vertical direction is set to a predetermined size, but it is not limited to this; the gap in the horizontal direction may also be set to a predetermined size, or the area of ​​region BB1 may also be set to a predetermined size.

[0064] Next, the control unit 1 performs a shake suppression display for the zoomed-in target TG (S63, processing as the display control unit 15).

[0065] As shown in Figure 9, in the shake suppression display, the range of the image GZ displayed on the screen SC is adjusted so that the area BB of the zoomed-in target TG is maintained at a predetermined position such as the center of the screen SC of the display unit 2A. Normally, zoomed-in moving images tend to shake, making them difficult for the user to see, but by performing shake suppression display, it is possible to suppress the appearance of shaking in the moving image of the target TG.

[0066] [Target Tracking Process] Figure 10 is a flowchart showing an example of the target tracking process procedure. Figures 11 and 12 are diagrams for explaining the target tracking process.

[0067] First, the control unit 1 acquires target data from the radar 3 (S21). The target data is TT data. The radar 3 tracks the detected echo image EC as the target TG and sequentially outputs TT data including the position and velocity of the target TG.

[0068] Next, the control unit 1 selects a target TG to be imaged from among the target TGs being tracked (S22). The selection of the target TG may be performed automatically, for example, according to the risk of collision, or it may be performed manually by the user UZ.

[0069] Next, the control unit 1 specifies the direction of the selected target TG (S14), causes the camera 5 to adjust the imaging direction (S15), and then zooms in (S16).

[0070] Next, the control unit 1 determines whether or not the target TG has been recognized in the zoomed-in image by the region recognition model (S23).

[0071] When a target TG is recognized (S23: YES), the control unit 1 sequentially updates the imaging direction of the camera 5A based on the position of the target TG included in the TT data sequentially output by the radar 3 (S24). This makes it possible to always display the tracking target TG on the display unit 2A.

[0072] On the other hand, if the target TG is not recognized (S23: NO), the control unit 1 determines that the selected target TG is a false image and excludes it from tracking (S25). This makes it possible to find echo images EC that are being tracked as target TG despite being false images.

[0073] [Second Designation Process] Figure 13 is a flowchart showing an example of the procedure for the first designation process. Figure 14 is a diagram for explaining the first designation process. In this example, camera 5B is a wide-angle camera with a wider field of view than camera 5A.

[0074] First, the control unit 1 acquires an image from the wide-angle camera 5B (S31), recognizes the region BB of the target TG in the image (S32, processing as the image recognition unit 12), and displays the image along with the recognition result (S33, processing as the display control unit 16).

[0075] Next, the control unit 1 receives the designation of the target TG by the user UZ (S13, processing as the operation reception unit 13). Specifically, the control unit 1 determines whether the area BB of the target TG displayed on the display unit 2B was designated by the user UZ's operation input.

[0076] Next, the control unit 1 specifies the direction of the selected target TG (S14), causes the camera 5A to adjust the imaging direction (S15), and performs zoom processing (S16). The direction of the target TG is calculated based on the imaging direction of the wide-angle camera 5B, the horizontal field of view of the camera 5B, and the horizontal position of the target TG in the image.

[0077] [Distance Estimation Process] Figure 15 is a flowchart showing an example of the distance estimation process. Figures 16 to 18 are diagrams for explaining the distance estimation process. In this example, both cameras 5A and 5B are PTZ cameras.

[0078] First, the control unit 1 uses cameras 5A and 5B to search for targets TG around the ship (S41). The search for targets TG is performed, for example, by panning cameras 5A and 5B and detecting targets TG using a region recognition model. The operation of the other camera 5A when a target TG is detected by one camera 5B will be described below.

[0079] As shown in Figure 16, when the target TG is located in the imaging direction VB of camera 5B, the region BB of the target TG is recognized in the center of the image GB generated by camera 5B. This state is considered to be the detection of the target TG.

[0080] When the camera 5B detects the target TG (S42: YES), the control unit 1 uses the camera 5A to search for the target TG in the imaging direction VB from the camera 5B toward the target TG (S43).

[0081] As shown in Figure 17, the search by camera 5A is performed by moving the imaging direction VA of camera 5A along the straight line of the imaging direction VB of camera 5B, from camera 5B toward the target TG.

[0082] Then, as shown in Figure 18, when the imaging direction VA of camera 5A reaches the target TG, that is, when the target TG is located on the imaging direction VA of camera 5A, the region BB of the target TG is recognized in the center of the image GA generated by camera 5A.

[0083] If camera 5A also detects the target TG (S44: YES), the control unit 1 calculates the distance from the ship to the target TG based on the imaging direction VA of camera 5A and the imaging direction VB of camera 5B (S45).

[0084] In other words, it becomes possible to calculate the distance from the ship to the target TG from the distance between camera 5A and camera 5B, the angle of the imaging direction VA from camera 5A toward the target TG, and the angle of the imaging direction VB from camera 5B toward the target TG.

[0085] Although embodiments of the present invention have been described above, the present invention is not limited to the embodiments described above, and various modifications are of course possible for those skilled in the art.

[0086] The following lists representative embodiments of the present invention.

[0087] (1) A navigation support system comprising: a target sensor for detecting the position of a target present around a vessel; a compass sensor for detecting the bow direction of the vessel; a camera installed on the vessel and having an attitude drive unit for changing the imaging direction; an imaging control unit for controlling the attitude drive unit to orient the imaging direction of the camera toward the direction of the target relative to the vessel; and a display control unit for displaying an image generated by the camera.

[0088] (2) The navigation support system according to (1), wherein the camera further has a lens drive unit that changes the focal length of the zoom lens, and the imaging control unit controls the lens drive unit to zoom in on the target.

[0089] (3) The navigation support system according to (1) or (2), wherein the target sensor is a radar or an AIS (Automatic Identification System), and the imaging control unit receives a designation of a target by the user and directs the imaging direction of the camera toward the direction of the designated target.

[0090] (4) The navigation support system according to (2), wherein the target sensor includes an image recognition unit that recognizes the region of the target in the image, and the imaging control unit controls the lens drive unit so that the region of the target is included in the image with a predetermined size.

[0091] (5) The navigation support system according to any one of (1) to (4), wherein the target sensor includes another camera installed on the vessel and an image recognition unit that recognizes the region of the target in an image generated by the other camera, the imaging control unit moves the imaging direction of the camera along a straight line from the other camera toward the target, and when the region of the target is recognized in an image generated by the camera, the system further includes a calculation unit that calculates the distance from the vessel to the target based on the direction from the camera toward the target and the direction from the other camera toward the target.

[0092] (6) The navigation support system according to any one of (1) to (5), wherein the target sensor is a radar, and further comprises: an image recognition unit that recognizes the target in the image generated by the camera, and a determination unit that determines that the target detected by the radar is a false image when the target is not recognized in the image.

[0093] (7) The navigation support system according to any one of (1) to (6), wherein the target sensor includes a wide-angle camera having a wider angle than the camera, and an image recognition unit that recognizes the region of the target in an image generated by the wide-angle camera, and the imaging control unit directs the imaging direction of the camera toward the direction of the target corresponding to the position of the region of the target in the image.

[0094] (8) The navigation support system according to any one of (1) to (7), wherein the target sensor is a radar that tracks the detected target, and the imaging control unit sequentially updates the imaging direction of the camera based on the position of the target output sequentially by the radar.

[0095] (9) The navigation support system according to any one of (1) to (8), wherein the image recognition unit recognizes the area of ​​the target in the image generated by the camera, and the display control unit adjusts the range of the image displayed on the screen so that the area of ​​the target is maintained in a predetermined position on the screen.

[0096] (10) A navigation support method comprising: detecting the position of a target present around a vessel using a target sensor; detecting the bow direction of the vessel using a compass sensor; controlling an attitude drive unit that changes the imaging direction of a camera installed on the vessel to orient the imaging direction of the camera toward the direction of the target relative to the vessel; and displaying an image generated by the camera.

[0097] (11) A program for causing a computer to perform the following actions: to obtain the position of a target present around the vessel as detected by a target sensor; to obtain the bow direction of the vessel as detected by a bearing sensor; to control the attitude drive unit of a camera installed on the vessel, which changes the imaging direction horizontally, so that the imaging direction of the camera is directed toward the target with respect to the vessel; and to display the image generated by the camera. term

[0098] Not all objectives or effects / benefits can necessarily be achieved in accordance with any particular embodiment described herein. Therefore, for example, a person skilled in the art will realize that a particular embodiment may be configured to achieve or optimize one or more effects / benefits taught herein, without necessarily achieving other objectives or effects / benefits taught or suggested herein.

[0099] All processes described herein can be fully automated and implemented by software code modules executed by a computing system including one or more computers or processors. The code modules can be stored in any type of non-temporary computer-readable medium or other computer storage device. Some or all of these methods can be implemented in dedicated computer hardware.

[0100] It will be apparent from this disclosure that there are many other variations not described herein. For example, depending on the embodiment, any particular operation, event, or function of any of the algorithms described herein may be performed in different sequences, and may be added, merged, or excluded entirely (e.g., not all described actions or events are necessary for the execution of the algorithm). Furthermore, in certain embodiments, the operations or events may be performed in parallel rather than sequentially, for example, through multithreading, interrupt handling, or via multiple processors or processor cores, or on other parallel architectures. In addition, different tasks or processes may also be performed by different machines and / or computing systems that can work together.

[0101] Various exemplary logic blocks and modules described in relation to the embodiments disclosed herein can be implemented or executed by a machine such as a processor. The processor may be a microprocessor, but alternatively, the processor may be a controller, a microcontroller, or a state machine, or a combination thereof. The processor may include electrical circuits configured to process computer-executable instructions. In another embodiment, the processor may include an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable device that performs logic operations without processing computer-executable instructions. The processor may also be implemented as a combination of computing devices, for example, a combination of a digital signal processor (digital signal processing device) and a microprocessor, multiple microprocessors, one or more microprocessors combined with a DSP core, or any other such configuration. Although the description herein mainly concerns digital technology, the processor may also include mainly analog elements. For example, some or all of the signal processing algorithms described herein may be implemented by analog circuits or mixed analog and digital circuits. The computing environment may include, but is not limited to, any type of computer system based on a microprocessor, mainframe computer, digital signal processor, portable computing device, device controller, or in-device computing engine.

[0102] Unless otherwise specified, conditional language such as “can,” “could,” “will,” or “may” is understood to mean in the context of commonly used expressions to convey that a particular embodiment includes certain features, elements, and / or steps, but other embodiments do not. Thus, such conditional language does not generally mean that features, elements, and / or steps are any way required in one or more embodiments, or that one or more embodiments necessarily include logic for determining whether these features, elements, and / or steps are included in or performed in any particular embodiment.

[0103] Disjunctive language, such as the phrase "at least one of X, Y, and Z," is understood in contexts where it is commonly used to indicate that an item, term, etc., can be any one of X, Y, Z, or any combination thereof, unless otherwise specified (e.g., X, Y, Z). Therefore, such disjunctive language does not generally imply that a particular embodiment requires each of at least one of X, at least one of Y, or at least one of Z, each of which exists.

[0104] Any process description, element, or block in the flowcharts described herein and / or shown in the accompanying drawings should be understood as representing a potentially module, segment, or portion of code containing one or more executable instructions for implementing a particular logical function or element in the process. Alternative embodiments are included within the scope of the embodiments described herein, where elements or functions may be removed, performed in no particular order, substantially simultaneously or in reverse order, depending on the relevant functionality, as will be understood by those skilled in the art.

[0105] Unless otherwise explicitly stated, numerals such as “one” should generally be interpreted as including one or more described items. Thus, phrases such as “one device configured to do…” are intended to include one or more enumerated devices. Such one or more enumerated devices may also be collectively configured to perform the stated citation. For example, “a processor configured to perform A, B and C below” could include a first processor configured to perform A and a second processor configured to perform B and C. In addition, even if an enumeration of a specific number of the introduced embodiments is explicitly listed, a person skilled in the art should interpret such an enumeration as typically meaning at least the number listed (for example, a mere enumeration of “two enumerations” without other modifiers usually means at least two enumerations, or two or more enumerations).

[0106] In general, a person skilled in the art will find that the terms used herein are generally intended to be "non-limiting" terms (for example, the term "including" should be interpreted as "including, but at least," the term "having" should be interpreted as "having at least," and the term "including" should be interpreted as "including, but not limited to, the following").

[0107] For illustrative purposes, the term “horizontal” as used herein is defined as a plane parallel to the floor or surface of the area in which the system described is used, or the plane on which the method described is implemented, regardless of its direction. The term “floor” may be replaced with the terms “ground” or “water surface.” The term “vertical / perpendicular” refers to a direction perpendicular / perpendicular to the defined horizontal line. Terms such as “upper side,” “lower side,” “below,” “up,” “on the side,” “higher,” “lower,” “above,” and “below” are defined in relation to the horizontal plane.

[0108] As used herein, the terms “adhere,” “connect,” “pair,” and other related terms should be interpreted, unless otherwise noted, as including removable, movable, fixed, adjustable, and / or removable connections or linkages. Connections / linkages include direct connections and / or connections having an intermediate structure between the two components described.

[0109] Unless otherwise explicitly stated, the numbers preceded by terms such as “approximately,” “about,” and “substantially,” as used herein, include the enumerated numbers and represent quantities close to the stated quantities that further perform the desired function or achieve the desired result. For example, “approximately,” “about,” and “substantially,” unless otherwise explicitly stated, mean values ​​less than 10% of the stated numbers. Features of embodiments disclosed preceded by terms such as “approximately,” “about,” and “substantially,” as used herein, represent features with some variability that further perform the desired function or achieve the desired result with respect to that feature.

[0110] Many variations and modifications can be made to the embodiments described above, and these elements should be understood as being within other acceptable examples. All such modifications and variations are intended to be within the scope of this disclosure and are protected by the following claims.

[0111] 1 Control unit, 2 (2A, 2B) Display unit, 3 Radar, 4 AIS, 5 (5A, 5B) Camera, 6 GNSS receiver, 7 Direction sensor, 8 ECDIS, 9 Wireless communication unit, 10 Ship handling control unit, 11, 12 Image recognition unit, 13 Operation reception unit, 14 Imaging command unit, 15, 16 Display control unit, 51 Imaging control unit, 52 Signal processing unit, 53 Pan drive unit, 54 Tilt drive unit, 55 Lens drive unit, 100 Navigation support system

Claims

1. A navigation support system comprising: a target sensor for detecting the position of targets present around a vessel; a compass sensor for detecting the bow direction of the vessel; a camera installed on the vessel and having an attitude drive unit for changing the imaging direction; an imaging control unit for controlling the attitude drive unit to orient the imaging direction of the camera toward the direction of the target relative to the vessel; and a display control unit for displaying the image generated by the camera.

2. The navigation support system according to claim 1, wherein the camera further comprises a lens drive unit that changes the focal length of the zoom lens, and the imaging control unit controls the lens drive unit to zoom in on the target.

3. The navigation support system according to claim 1, wherein the target sensor is a radar or an AIS (Automatic Identification System), and the imaging control unit receives a designation of a target by the user and directs the imaging direction of the camera toward the direction of the designated target.

4. The navigation support system according to claim 2, wherein the target sensor includes an image recognition unit that recognizes the region of the target in the image, and the imaging control unit controls the lens drive unit so that the region of the target is included in the image with a predetermined size.

5. The navigation support system according to claim 1, wherein the target sensor includes another camera installed on the vessel and an image recognition unit that recognizes the region of the target in an image generated by the other camera, the imaging control unit moves the imaging direction of the camera along a straight line from the other camera toward the target, and when the region of the target is recognized in an image generated by the camera, the system further includes a calculation unit that calculates the distance from the vessel to the target based on the direction from the camera toward the target and the direction from the other camera toward the target.

6. The navigation support system according to claim 1, wherein the target sensor is a radar, and further comprises: an image recognition unit that recognizes the target in the image generated by the camera, and a determination unit that, when the target is recognized in the image, determines that the target detected by the radar is not a false image.

7. The navigation support system according to claim 1, wherein the target sensor includes a wide-angle camera having a wider angle than the camera, and an image recognition unit that recognizes the region of the target in an image generated by the wide-angle camera, and the imaging control unit directs the imaging direction of the camera toward the direction of the target corresponding to the position of the region of the target in the image.

8. The navigation support system according to claim 1, wherein the target sensor is a radar that tracks the detected target, and the imaging control unit sequentially updates the imaging direction of the camera based on the position of the target sequentially output by the radar.

9. The navigation support system according to claim 1, comprising: an image recognition unit that recognizes the region of the target in the image generated by the camera; and a display control unit that adjusts the range of the image displayed on the screen so that the region of the target is maintained in a predetermined position on the screen.

10. A navigation assistance method comprising: detecting the position of a target present around a vessel using a target sensor; detecting the bow direction of the vessel using a compass sensor; controlling an attitude drive unit that changes the imaging direction of a camera installed on the vessel to orient the imaging direction of the camera toward the direction of the target relative to the vessel; and displaying an image generated by the camera.

11. A program to cause a computer to perform the following actions: acquire the position of targets present around a vessel as detected by a target sensor; acquire the bow direction of the vessel as detected by a compass sensor; control the attitude drive unit of a camera installed on the vessel to change the imaging direction horizontally, so that the imaging direction of the camera is directed toward the target relative to the vessel; and display the image generated by the camera.

Images