Navigation support systems, navigation support methods, and programs

Abstract

This provides a navigation support system capable of determining the camera's imaging direction. [Solution] The navigation support system is installed on a ship and includes a camera that acquires images including targets at sea, an image recognition unit that recognizes the area of ​​the target in the image, a compass sensor that detects the ship's bow direction, a target sensor that detects the position of the target, and a calculation unit that calculates the difference between the camera's imaging direction and the ship's bow direction based on the horizontal image position of the target in the image and the direction of the target relative to the ship.

Description

Technical Field

[0001] The present invention relates to a navigation support system, a navigation support method, and a program.

Background Art

[0002] Conventionally, a technique for detecting a maritime target such as a ship in an image captured by a camera mounted on a ship by image recognition is known.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] By the way, a camera mounted on a ship is installed, for example, facing the bow direction, but there may be a case where the imaging direction of the camera is deviated from the intended direction. In that case, there is a risk that the direction of the target detected by image recognition cannot be correctly grasped.

[0005] The present invention has been made in view of the above problems, and its main object is to provide a navigation support system, a navigation support method, and a program capable of specifying the imaging direction of a camera.

Means for Solving the Problems

[0006] To solve the above problems, a navigation support system according to one aspect of the present invention includes a camera installed on a ship that acquires an image including a target at sea, an image recognition unit that recognizes the region of the target in the image, a compass sensor that detects the ship's bow direction, a target sensor that detects the position of the target, and a calculation unit that calculates the difference between the camera's imaging direction and the ship's bow direction based on the horizontal image position of the target in the image and the direction of the target relative to the ship. This makes it possible to specify the camera's imaging direction.

[0007] In the above embodiment, the camera may have a drive unit that changes the imaging direction in the horizontal direction. This makes it possible to specify the imaging direction of the camera even if the camera is capable of changing the imaging direction in the horizontal direction.

[0008] In the above embodiment, the calculation unit may calculate the imaging direction of the camera with respect to the bow direction. This makes it possible to determine the imaging direction of the camera with respect to the bow direction.

[0009] In the above embodiment, the system may further include an identification unit that identifies the object recognized in the image with the object detected by the object sensor. This makes it possible to identify the object recognized in the image with the object detected by the object sensor.

[0010] In the above embodiment, the identification unit may identify the multiple targets based on the horizontal appearance pattern of the multiple targets in the image and the circumferential appearance pattern of the multiple targets relative to the vessel. This makes it possible to identify multiple targets.

[0011] In the above embodiment, the image recognition unit may further recognize the size of the target, the target sensor may further detect the size of the target, and the identification unit may further identify the target based on the size of the target. This makes it possible to improve the accuracy of target identification.

[0012] In the above embodiment, the image recognition unit may further recognize the orientation of the target, the target sensor may further detect the orientation of the target, and the identification unit may further identify the target based on the orientation of the target. This makes it possible to improve the accuracy of target identification.

[0013] In the above embodiment, the image recognition unit may further recognize the type of the target, the target sensor may further detect the type of the target, and the identification unit may further identify the target based on the type of target. This makes it possible to improve the accuracy of target identification.

[0014] In the above embodiment, the target sensor may be a radar. This makes it possible to determine the imaging direction of the camera by utilizing the position of the target detected by the radar.

[0015] In the above embodiment, the target sensor may be an AIS (Automatic Identification System). This makes it possible to determine the camera's imaging direction by utilizing the position of the target detected by the AIS.

[0016] Furthermore, in another embodiment of the present invention, a navigation support method involves acquiring an image including a target at sea using a camera installed on a ship, recognizing the area of ​​the target in the image, detecting the ship's bow direction using a compass sensor, detecting the position of the target using a target sensor, and calculating the difference between the camera's imaging direction and the ship's bow direction based on the horizontal image position of the target in the image and the direction of the target relative to the ship. This makes it possible to determine the camera's imaging direction.

[0017] In addition, a program according to another aspect of the present invention causes a computer to acquire an image including a maritime target generated by a camera installed on a ship, recognize a region of the target in the image, acquire the bow direction of the ship detected by a direction sensor, acquire the position of the target detected by a target sensor, and calculate a difference between the imaging direction of the camera and the bow direction based on the in-image position of the target in the horizontal direction in the image and the direction of the target with respect to the ship. According to this, it becomes possible to specify the imaging direction of the camera.

Advantages of the Invention

[0018] According to the present invention, it becomes possible to specify the imaging direction of the camera.

Brief Description of the Drawings

[0019] [Figure 1] It is a diagram showing an example of a system. [Figure 2] It is a diagram showing an example of a control unit. [Figure 3] It is a diagram showing an example of a camera. [Figure 4] It is a diagram showing an example of the relationship between the bow direction and the imaging direction. [Figure 5] It is a diagram showing an example of a target database for a camera. [Figure 6] It is a diagram showing an example of a target database for sensors. [Figure 7] It is a diagram for explaining the identification of a target and the calibration of the imaging direction. [Figure 8] It is a diagram for explaining the identification of a target and the calibration of the imaging direction. [Figure 9] It is a diagram for explaining the identification of a target and the calibration of the imaging direction. [Figure 10] It is a diagram showing an example of a navigation support method.

Embodiments for Carrying Out the Invention

[0020] Embodiments of the present invention will be described below with reference to the drawings. In this specification and in each drawing, elements similar to those described above in relation to previously shown drawings will be denoted by the same reference numerals, and detailed descriptions may be omitted as appropriate.

[0021] Figure 1 is a block diagram showing an example configuration of the navigation support system 100. The navigation support system 100 is a system installed on a ship. In the following description, a ship equipped with the navigation support system 100 will be referred to as "our ship," and other ships will be referred to as "other ships."

[0022] The navigation support system 100 comprises a control unit 1, a display unit 2, a radar 3, an AIS 4, a camera 5, a GNSS receiver 6, a direction sensor 7, an ECDIS 8, a wireless communication unit 9, and a ship handling control unit 10. These devices are connected to a network N, such as a LAN, and are capable of network communication with each other.

[0023] The control unit 1 includes a computer comprising a CPU, RAM, ROM, non-volatile memory, and an input / output interface. The CPU of the control unit 1 performs information processing according to a program loaded from ROM or non-volatile memory into RAM.

[0024] The program may be supplied via an information storage medium such as an optical disc or memory card, or via a communication network such as the Internet or LAN.

[0025] The display unit 2 displays the display image generated by the control unit 1. The display unit 2 can also display radar images, camera images, or electronic charts.

[0026] The display unit 2 is, for example, a display device with a touch sensor, a so-called touch panel, which detects the position indicated on the screen by the user's finger or the like. However, it is not limited to this, and the indicated position may also be input by a pointing device such as a trackball.

[0027] Radar 3 emits radio waves around the ship and receives the reflected waves, generating echo data based on the received signals. Radar 3 also identifies targets from the echo data and generates TT data (Target Tracking Data) representing the target's position and speed.

[0028] The Automatic Identification System (AIS) 4 receives AIS data from other vessels or shore-based control systems in the vicinity of the vessel. While AIS is not the only option, a VHF Data Exchange System (VDES) may also be used. AIS data includes the identification code, name, position, course, speed, vessel type, length, and destination of other vessels.

[0029] Camera 5 is a digital camera that captures images of the outside from the ship and generates image data. Camera 5 is installed, for example, on the ship's bridge, facing the bow. Camera 5 is a so-called PTZ camera, which has, for example, pan-tilt and optical zoom functions.

[0030] Camera 5 may include an image recognition unit that estimates the position and type of objects at sea, such as other vessels, contained in the captured image using an object detection model. The image recognition unit is not limited to camera 5 and may be implemented in other devices such as control unit 1.

[0031] The GNSS receiver 6 detects the ship's position based on radio waves received from the GNSS (Global Navigation Satellite System). The bow sensor 7 is, for example, a GPS compass or a gyrocompass, and detects the ship's bow direction.

[0032] The ECDIS (Electronic Chart Display and Information System) 8 obtains the ship's position from the GNSS receiver 6 and displays the ship's position on the electronic chart. The ECDIS 9 also displays the ship's planned route on the electronic chart. A GNSS plotter may be used instead of the ECDIS.

[0033] The wireless communication unit 9 includes wireless equipment for realizing satellite communications. The wireless communication unit 9 also includes wireless equipment for realizing ship-to-shore or ship-to-ship wireless communications using, for example, ultra-high frequency, very high frequency, short frequency, or medium-high frequency.

[0034] The ship steering control unit 10 is a control device for achieving autonomous navigation and controls the ship's steering gear. The ship steering control unit 10 may also control the ship's engines.

[0035] In this embodiment, the control unit 1 and the display unit 2 are independent devices, but the control unit 1 and the display unit 2 may be an integrated device.

[0036] Furthermore, the control unit 1 is not limited to an independent device, but may be integrated with other devices such as ECDIS8. In other words, some or all of the functions of the control unit 1 may be implemented by other devices.

[0037] Furthermore, the display unit 2 is not limited to an independent device; the display unit of another device, such as ECDIS8, may be used as the display unit 2 that displays the display image generated by the control unit 1.

[0038] Furthermore, the control unit 1 and the display unit 2 may be installed, for example, at a land-based control center and used to monitor vessels navigating within a controlled area.

[0039] Figure 2 is a block diagram showing an example configuration of the control unit 1. The control unit 1 comprises an image recognition unit 11, a data acquisition unit 12, a data acquisition unit 13, a target identification unit 14, and an imaging direction calculation unit 15. These functional units are realized by the CPU of the control unit 1 executing information processing according to a program.

[0040] Furthermore, the control unit 1 can access the camera target database 22 and the sensor target database 23. These databases 22 and 23 may be stored in the memory of the control unit 1 or in an external storage device.

[0041] Figure 3 is a block diagram showing an example configuration of camera 5. Camera 5 is a PTZ camera having pan-tilt and optical zoom functions, and comprises an imaging control unit 51, a signal processing unit 52, a pan drive unit 53, a tilt drive unit 54, and a lens drive unit 55.

[0042] The pan drive unit 53 changes the imaging direction of the camera 5 horizontally, and the tilt drive unit 54 changes the imaging direction of the camera 5 vertically. The pan drive unit 53 and the tilt drive unit 54 are examples of attitude control units. The lens drive unit 55 changes the focal length of the zoom lens 50 to zoom in or out on the subject.

[0043] Figure 4 shows an example of the relationship between the bow direction BW of the vessel SH and the imaging direction MD of camera 5. Camera 5 images the area around the vessel SH while changing the imaging direction MD horizontally, i.e., panning.

[0044] Incidentally, camera 5 is installed so that the reference position of the imaging direction MD (for example, the center of the movable range) faces the bow direction BW, etc., but in reality, there may be errors in the imaging direction MD. If there are errors in the imaging direction MD, there is a risk that the position of targets present around the ship SH will not be accurately determined.

[0045] Therefore, in this embodiment, as described below, the imaging direction MD of the camera 5 is calibrated by utilizing data detected by a target sensor such as radar 3 or AIS 4.

[0046] Returning to the explanation of Figure 2, the image recognition unit 11 acquires images generated by the camera 5 installed on the ship's SH, and inputs the acquired images into the region recognition model to recognize the region of the target in the image. The region recognition model is a pre-trained model obtained through machine learning. However, it is not limited to this; the region recognition model may also be a rule-based computation model.

[0047] The region recognition model is, for example, an object detection model such as YOLO (You Only Look Once) or SSD (Single Shot MultiBox Detector), which outputs a bounding box surrounding objects in the image. However, the region recognition model is not limited to these; it may also be a region segmentation model such as Semantic Segmentation or Instance Segmentation.

[0048] The data acquisition unit 12 acquires target data of targets included in the image based on the image recognition results from the image recognition unit 11 and registers it in the camera target DB 22. The target data includes the direction of the target relative to the ship SH.

[0049] As shown in Figure 5, the camera target DB22 includes fields such as "ID," "Source," "Direction," "Size," "Orientation," and "Type."

[0050] "ID" is the identifier of the target recognized from the image. "Source" indicates that the target data was acquired by camera 5.

[0051] "Direction" refers to the direction of the target relative to the vessel SH. The direction of the target is calculated based on the imaging direction MD of camera 5, the horizontal field of view of camera 5, and the horizontal position of the target in the image.

[0052] The imaging direction MD of camera 5 is calculated, for example, by obtaining the rotation angle of the pan drive unit 53 of camera 5. If camera 5 does not have a pan function, the imaging direction MD of camera 5 will be a fixed value.

[0053] "Size" refers to the size of the target. The target size is calculated based on the size of the target's region within the image. The target's region is the bounding box surrounding the target.

[0054] "Orientation" refers to the orientation of the target. The orientation of the target may be, for example, the target's course calculated based on the temporal change in the target's position within the image, or it may be the target's heading (ship's bearing) identified from the image of the target.

[0055] "Type" represents the type of target. The target type is identified from the target image by, for example, a trained model. The target type is, for example, the type of vessel such as a pleasure boat, fishing boat, merchant ship, or tanker.

[0056] The camera target DB22 may further include the target's position and velocity. The target's position is calculated, for example, based on the target's direction and the distance from the ship to the target estimated by a trained model, and the target's velocity is calculated based on the change in the target's position over time.

[0057] The data acquisition unit 13 acquires target data of targets detected by target sensors such as radar 3 or AIS 4, and registers it in the sensor target DB 23. The target data is, for example, TT data or AIS data, and includes the position of the target.

[0058] As shown in Figure 6, the sensor target DB23 includes fields such as "ID", "Source", "Position", "Velocity", "Direction", "Size", "Orientation", and "Type".

[0059] "ID" is the identifier of the target detected by Radar 3 or AIS 4. "Source" indicates which target sensor the target data was acquired from.

[0060] "Position" represents the location of the target. The target's position is expressed in absolute coordinates of latitude and longitude. In the case of radar 3, the target's position included in the TT data is a relative position with respect to the ship's SH, so it is converted to absolute coordinates using the ship's SH position detected by the GNSS receiver 6. In the case of AIS 4, the target's position corresponds to the position included in the AIS data.

[0061] "Speed" represents the velocity of a target. In the case of Radar 3, the velocity of a target is calculated from the change in the target's position over time. In the case of AIS 4, the velocity of a target corresponds to the ship's speed and course included in the AIS data. Speed ​​is a vector quantity that includes ship's speed and course or heading.

[0062] "Direction" refers to the direction of the target relative to the ship's SH (Sailhead Wing). In the case of radar 3, the direction of the target is determined based on the ship's SH's bow direction BW detected by the direction sensor 7. In the case of AIS 4, the direction of the target is calculated based on the target's position and the ship's SH's bow direction BW detected by the direction sensor 7.

[0063] "Size" refers to the size of the target. In the case of Radar 3, the target size corresponds to the size of the echo image. In the case of AIS 4, the target size corresponds to the hull length included in the AIS data.

[0064] "Orientation" refers to the orientation of the target. In the case of Radar 3, the orientation of the target is represented by the target's trajectory included in the TT data. In the case of AIS 4, the orientation of the target is represented by the heading included in the AIS data.

[0065] "Type" represents the type of target. In the case of Radar 3, the type of target cannot be determined in principle, but it is possible to estimate the type of target from the echo image using a trained model. In the case of AIS 4, the type of target corresponds to the type included in the AIS data.

[0066] Returning to the explanation of Figure 2, the target identification unit 14 identifies targets registered in the camera target DB 22 and targets registered in the sensor target DB 23. Specifically, the target identification unit 14 determines whether the targets are the same by comparing the direction of the target registered in the camera target DB 22 with the direction of the target registered in the sensor target DB 23.

[0067] The imaging direction calculation unit 15 calculates the difference θ between the imaging direction MD of the camera 5 and the bow direction BW based on the direction of the target registered in the camera target DB 22 and the direction of the target registered in the sensor target DB 23, and also calculates the imaging direction MD of the camera 5 with respect to the bow direction BW.

[0068] Figures 7 and 8 illustrate the identification of targets TG1 to TG3 by the target identification unit 14 and the calibration of the camera 5's imaging direction MD by the imaging direction calculation unit 15. In Figure 9, for reference, vertical lines representing the directions BD and CD of the target TG are shown within image G.

[0069] Targets TG1 to TG3 are located around the vessel SH, and image G, captured by camera 5 mounted on the vessel SH, includes targets TG1 to TG3. Furthermore, image G shows boundary boxes BB1 to BB3 surrounding targets TG1 to TG3, as recognized by the region recognition model.

[0070] The target directions BD1 to BD3 based on camera 5 are calculated based on the horizontal in-image positions of the bounding boxes BB1 to BB3 in image G. Specifically, the directions BD1 to BD3 are calculated based on the imaging direction of camera 5, the horizontal field of view of camera 5, and the horizontal in-image positions of the bounding boxes BB1 to BB3 in image G.

[0071] On the other hand, the target directions CD1 to CD3 based on radar 3 or AIS 4 are calculated based on the ship's bow direction BW detected by the heading sensor 7. Directions CD1 to CD3 correctly represent the directions of targets TG1 to TG3.

[0072] Figure 7 shows an example where the target directions BD1 to BD3 were calculated assuming that camera 5 was facing the bow direction BW, but in reality camera 5 is facing a different direction from the bow direction BW.

[0073] In this case, targets TG1 to TG3 do not exist on the target directions BD1 to BD3 based on camera 5. In other words, the target directions BD1 to BD3 based on camera 5 are offset from the target directions CD1 to CD3 based on radar 3 or AIS 4.

[0074] The target identification unit 14 identifies targets TG1 to TG3 by comparing the target directions BD1 to BD3 based on the camera 5 with the target directions CD1 to CD3 based on the radar 3 or AIS 4.

[0075] Specifically, the target identification unit 14 searches for a rotation angle in which the appearance pattern of target directions BD1 to BD3 based on the camera 5 matches the appearance pattern of target directions CD1 to CD3 based on the radar 3 or AIS 4.

[0076] Furthermore, the target identification unit 14 may identify targets TG1 to TG3 by comparing not only the direction of the targets, but also one or more of the following: the size, orientation, and type of the targets. This makes it possible to improve the accuracy of target identification.

[0077] Figure 8 shows an example where the target directions BD1-BD3 based on camera 5 are rotated until they match the target directions CD1-CD3 based on radar 3 or AIS 4.

[0078] The imaging direction calculation unit 15 determines the actual imaging direction MD of the camera 5 by rotating the target directions BD1 to BD3 based on the camera 5 until they match the target directions CD1 to CD3 based on the radar 3 or AIS 4. In other words, the imaging direction calculation unit 15 works backward from the directions BD1 to BD3 rotated to match the directions CD1 to CD3 to determine the actual imaging direction MD of the camera 5.

[0079] The actual imaging direction MD of camera 5 is represented by the difference θ between the imaging direction MD of camera 5 and the bow direction BW, and is calculated as the imaging direction MD of camera 5 relative to the bow direction BW.

[0080] The difference θ between the imaging direction MD of camera 5 and the bow direction BW is calculated based on the rotation angles of directions BD1 to BD3, which coincide with directions CD1 to CD3. In this example, since directions BD1 to BD3 were calculated assuming that camera 5 was initially facing the bow direction BW, the rotation angles of directions BD1 to BD3, which coincide with directions CD1 to CD3, are equal to the difference θ between the imaging direction MD of camera 5 and the bow direction BW.

[0081] Figure 10 is a flowchart showing an example of the procedure for calibrating the imaging direction MD of camera 5, which is one of the navigation support methods implemented in the navigation support system 100. The control unit 1 executes the information processing shown in the figure according to the program.

[0082] First, the control unit 1 acquires the imaging direction of the camera 5 (S11). Here, the imaging direction of the camera 5 before calibration is acquired. Next, the control unit 1 acquires the image G captured by the camera 5 (S12).

[0083] Next, the control unit 1 inputs the acquired image G into the region recognition model, thereby recognizing the targets TG1 to TG3 within the image G using boundary boxes BB1 to BB3 (S13, processing by the image recognition unit 11).

[0084] Next, the control unit 1 detects the directions B1 to B3 of the target relative to its own vessel SH based on the recognition result of image G (S14, processing as the data acquisition unit 12).

[0085] Meanwhile, the control unit 1 detects the bow direction BW of the vessel SH using the heading sensor 7 (S15). Next, the control unit 1 acquires target data from the radar 3 or AIS 4 (S16, processing as the data acquisition unit 13).

[0086] Next, the control unit 1 detects the target directions C1 to C3 relative to the ship's SH based on target data acquired from radar 3 or AIS 4 (S17, processing as data acquisition unit 13).

[0087] The control unit 1 then identifies the targets TG1 to TG3 recognized from the image G with the targets TG1 to TG3 detected by the radar 3 or AIS 4 (S18, processing as the target identification unit 14).

[0088] Next, the control unit 1 calculates the difference θ between the actual imaging direction MD of the camera 5 and the bow direction BW, and determines the imaging direction MD of the camera 5 based on this (S19, S20, processing as the imaging direction calculation unit 15).

[0089] With the above steps completed, the series of processes for calibrating the imaging direction MD of camera 5 is finished.

[0090] Although embodiments of the present invention have been described above, the present invention is not limited to the embodiments described above, and various modifications are of course possible for those skilled in the art.

[0091] The following lists representative embodiments of the present invention.

[0092] (1) A camera installed on a ship to acquire images including objects at sea, An image recognition unit that recognizes the region of the target in the aforementioned image, A compass sensor for detecting the bow direction of the aforementioned vessel, A target sensor for detecting the position of the target, A calculation unit calculates the difference between the camera's imaging direction and the ship's bow direction based on the horizontal position of the target in the image and the direction of the target relative to the ship. A navigation support system equipped with [the following features].

[0093] (2) The camera has a drive unit that changes the imaging direction in the horizontal direction. (1) The navigation support system described in (1).

[0094] (3) The calculation unit calculates the imaging direction of the camera with respect to the bow direction of the ship. The navigation support system described in (1) or (2).

[0095] (4) The system further includes an identification unit that identifies the object recognized in the aforementioned image with the object detected by the object sensor. A navigation support system as described in any of (1) to (3).

[0096] (5) The identification unit identifies the multiple targets based on the horizontal appearance pattern of the multiple targets in the image and the circumferential appearance pattern of the multiple targets relative to the vessel. (4) Navigation support system as described above.

[0097] (6) The image recognition unit further recognizes the size of the target, The target sensor further detects the size of the target, The identification unit further identifies the target based on the size of the target. A navigation support system as described in any of (1) through (5).

[0098] (7) The image recognition unit further recognizes the orientation of the target, The target sensor further detects the orientation of the target, The identification unit further identifies the target based on the orientation of the target. A navigation support system as described in any of (1) through (6).

[0099] (8) The image recognition unit further recognizes the type of the target, The target sensor further detects the type of the target, The identification unit further identifies the target based on the type of the target. A navigation support system as described in any of (1) through (7).

[0100] (9) The aforementioned target sensor is a radar. A navigation support system as described in any of (1) through (8).

[0101] (10) The aforementioned target sensor is an AIS (Automatic Identification System). A navigation support system as described in any of (1) through (9).

[0102] (11) Cameras installed on the ship acquire images including objects at sea. Recognizing the area of ​​the target in the aforementioned image, The direction of the bow of the ship is detected by the compass sensor, The target sensor detects the position of the target, Based on the horizontal position of the target in the image and the direction of the target relative to the vessel, the difference between the camera's imaging direction and the ship's bow direction is calculated. Navigation aid methods.

[0103] (12) To acquire images, including objects at sea, generated by cameras installed on ships. To recognize the region of the target in the aforementioned image, To obtain the direction of the bow of the ship detected by the direction sensor, To acquire the position of the target detected by the target sensor, and Based on the horizontal position of the target in the image and the direction of the target relative to the vessel, the difference between the camera's imaging direction and the ship's bow direction is calculated. A program that causes a computer to execute something. [Explanation of Symbols]

[0104] 1 Control unit, 2 Display unit, 3 Radar, 4 AIS, 5 Camera, 6 GNSS receiver, 7 Direction sensor, 8 ECDIS, 9 Wireless communication unit, 10 Ship handling control unit, 11 Image recognition unit, 12 Data acquisition unit, 13 Data acquisition unit, 14 Target identification unit, 15 Image acquisition direction calculation unit, 100 Navigation support system

Claims

1. A camera installed on a ship to acquire images including objects at sea, An image recognition unit that recognizes the region of the target in the aforementioned image, A compass sensor for detecting the bow direction of the aforementioned vessel, A target sensor for detecting the position of the target, A calculation unit calculates the difference between the camera's imaging direction and the ship's bow direction based on the horizontal position of the target in the image and the direction of the target relative to the ship. A navigation support system equipped with [the following features].

2. The camera has a drive unit that changes the imaging direction in the horizontal direction. The navigation support system according to claim 1.

3. The calculation unit calculates the imaging direction of the camera with respect to the bow direction of the ship. The navigation support system according to claim 1.

4. The system further includes an identification unit that identifies the object recognized in the aforementioned image with the object detected by the object sensor. The navigation support system according to claim 1.

5. The identification unit identifies the multiple targets based on the horizontal appearance pattern of the multiple targets in the image and the circumferential appearance pattern of the multiple targets relative to the vessel. The navigation support system according to claim 4.

6. The image recognition unit further recognizes the size of the target, The target sensor further detects the size of the target, The identification unit further identifies the target based on the size of the target. The navigation support system according to claim 1.

7. The image recognition unit further recognizes the orientation of the target, The target sensor further detects the orientation of the target, The identification unit further identifies the target based on the orientation of the target. The navigation support system according to claim 1.

8. The image recognition unit further recognizes the type of the target, The target sensor further detects the type of the target, The identification unit further identifies the target based on the type of the target. The navigation support system according to claim 1.

9. The aforementioned target sensor is a radar. The navigation support system according to claim 1.

10. The aforementioned target sensor is an AIS (Automatic Identification System). The navigation support system according to claim 1.

11. Cameras installed on the ship acquire images including objects at sea. Recognizing the area of ​​the target in the aforementioned image, The direction of the bow of the ship is detected by the compass sensor, The target sensor detects the position of the target, Based on the horizontal position of the target in the image and the direction of the target relative to the vessel, the difference between the camera's imaging direction and the ship's bow direction is calculated. Navigation aid methods.

12. To acquire images, including objects at sea, generated by cameras installed on ships. To recognize the region of the target in the aforementioned image, To obtain the direction of the bow of the ship detected by the direction sensor, To acquire the position of the target detected by the target sensor, and Based on the horizontal position of the target in the image and the direction of the target relative to the vessel, the difference between the camera's imaging direction and the ship's bow direction is calculated. A program that causes a computer to execute something.

Images