Monitoring system and monitoring method of ship

Abstract

To provide a monitoring system and a monitoring method of a ship capable of more surely grasping positional information of a monitored target type-ship existing in a monitored target water area with less time lag.SOLUTION: A monitoring system inputs image data D1 in a surrounding water area SW acquired by a camera device 2 installed at a monitoring reference position M in a calculation device 3 and stores a detection model to detect the image of a monitored target type-ship 20 in the calculation device 3 in advance. The monitoring system detects the image of the monitored target type-ship 20 in the image data D1 using the calculation device 3 based on the detection model and the image data D1, and displays the detection frame F representing its image range on the image data D1. The monitoring system displays a grid G representing the relative position to a monitoring reference position on the water surface of the image data D1. The monitoring system detects the positional information of the monitored target type-ship 20 based on the relationship between the detection frame F in the image data D1 and the grid G, and displays the positional information of the monitored target type-ship 20 on monitors 6, 7a.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a ship monitoring system and monitoring method, and more specifically, to a ship monitoring system and monitoring method that can more reliably obtain position information of a ship of a monitored type that is present in a monitored water area with less time lag. [Background technology]

[0002] When monitoring the surrounding waters from ships or land-based construction sites facing water, ship crew members and workers at the construction site basically use binoculars to visually monitor ships in the surrounding waters. However, with visual monitoring, it is difficult to accurately grasp the position of the ship of the type to be monitored in the surrounding waters, and in particular, it is difficult to accurately grasp the relative distance between the position being monitored (monitoring reference position) and the ship of the type to be monitored. There is also a concern that with visual monitoring, monitoring workers may overlook ships of the type to be monitored that are in the surrounding waters.

[0003] In order to detect ships of the type to be monitored that are present in the surrounding waters, for example, an AIS device (see, for example, Patent Document 1) or a radar device can be used. However, since ships of a gross tonnage of 499 tons or less (small vessels such as cargo handling ships, fishing boats, and pleasure boats) are not required to be equipped with an AIS device, AIS devices cannot detect ships of a gross tonnage of 499 tons or less. Although AIS information includes ship position information, the update frequency of the AIS information transmitted from the AIS device is determined according to the speed of the ship that is equipped with the AIS device. Therefore, if the speed of the ship equipped with the AIS device is slow, the update frequency of the AIS information is at time intervals of about several seconds, and it is not possible to continuously grasp the position information of the type of ship to be monitored in real time. In addition, while a radar device is suitable for detecting ships that are relatively far away, it has a poor detection range in a short distance range (for example, within one nautical mile). Therefore, there is room for improvement in grasping the position information of the type of ship to be monitored that is present in the waters to be monitored more reliably with less time lag. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2013-116686 A Summary of the Invention [Problem to be solved by the invention]

[0005] An object of the present invention is to provide a ship monitoring system and monitoring method that can obtain position information of a ship of a monitored type that is present in a monitored water area more reliably and with less time lag. [Means for solving the problem]

[0006] In order to achieve the above object, the ship monitoring system of the present invention comprises a camera device that is installed at a predetermined monitoring reference position and sequentially acquires image data of the surrounding water area of ​​the monitoring reference position, a calculation device to which the image data of the surrounding water area acquired by the camera device is sequentially input, and a monitor communicably connected to the calculation device, the calculation device has a memory in which a detection model for detecting images of a ship of a monitored type that has been machine-learned using image data of multiple types of ships and image data of non-ships as training data is stored, and a calculation processing unit that performs calculation processing, and the calculation processing unit is configured to perform a calculation process based on the detection model and the image data of the surrounding water area input from the camera device. Based on this, the system is configured to execute a detection process which detects images of the predetermined type of ship to be monitored that is captured in image data of the surrounding waters, and superimposes a detection frame indicating the range of the detected image of the ship of the type to be monitored on the image data of the surrounding waters; a grid display process which superimposes a grid indicating a relative position to the monitoring reference position on the water surface in the image data of the surrounding waters; and a position information display process which calculates position information of the ship of the type to be monitored based on the positional relationship between the detection frame and the grid in the image data of the surrounding waters, and displays the calculated position information of the ship of the type to be monitored on the monitor.

[0007] The ship monitoring method of the present invention is characterized in that a camera device is installed at a predetermined monitoring reference position, which sequentially acquires image data of the surrounding waters of the monitoring reference position, the image data of the surrounding waters acquired by the camera device is sequentially input to a computing device, and a detection model for detecting images of ships of a monitored type that has been machine-learned using image data of multiple types of ships and image data of non-ships as training data is stored in the computing device in advance, and the computing device is used to detect images of ships of a predetermined monitored type that are reflected in the image data of the surrounding waters based on the detection model and the image data of the surrounding waters input from the camera device, and a detection frame indicating the range of the detected images of ships of the monitored type are superimposed on the image data of the surrounding waters, a grid indicating a relative position with respect to the monitoring reference position is superimposed on the water surface in the image data of the surrounding waters, and position information of the ships of the monitored type is calculated based on the positional relationship between the detection frame and the grid in the image data of the surrounding waters, and the calculated position information of the ships of the monitored type is displayed on a monitor. Effect of the Invention

[0008] According to the present invention, the camera device installed at a predetermined monitoring reference position sequentially acquires image data of the surrounding water area of ​​the monitoring reference position, and the acquired image data of the surrounding water area is sequentially input to the calculation device. Then, a detection model for detecting images of ships of a monitoring target type that has been machine-learned using image data of multiple types of ships and image data of non-ships as teacher data is stored in the calculation device, and the calculation device detects images of ships of a preset monitoring target type reflected in the image data of the surrounding water area based on the detection model and the image data of the surrounding water area, and a detection frame indicating the range of the detected image of the ship of the monitoring target type is superimposed on the image data of the surrounding water area. Furthermore, by superimposing and displaying a grid indicating a relative position with respect to the monitoring reference position on the water surface in the image data of the surrounding water area, position information of the monitoring target type of ship can be calculated based on the positional relationship between the detection frame and the grid in the image data of the surrounding water area. Then, by displaying the calculated position information of the monitoring target type of ship on a monitor, it is possible to grasp the position information of the monitoring target type of ship present in the monitoring target water area (surrounding water area) more reliably and with less time lag. [Brief description of the drawings]

[0009] [Figure 1] 1 is an explanatory diagram illustrating an embodiment of a ship monitoring system of the present invention as seen from the side; [Diagram 2] FIG. 2 is an explanatory diagram illustrating a schematic configuration of the monitoring system of FIG. [Diagram 3] FIG. 2 is an explanatory diagram illustrating image data of the surrounding water area acquired by a camera device. [Figure 4] FIG. 11 is an explanatory diagram illustrating an example of image data displayed on a work monitor. [Diagram 5] 2 is an explanatory diagram illustrating the monitoring system of FIG. 1 in a plan view. [Figure 6] FIG. 13 is an explanatory diagram illustrating an example of monitoring image data displayed on a monitoring monitor; [Figure 7] FIG. 13 is an enlarged view of a portion of image data displayed on a work monitor during monitoring. [Figure 8]It is an enlarged view of a part of another image data displayed on a work monitor during monitoring. [Figure 9] It is an explanatory diagram illustrating in side view the situation where the calibration of the monitoring system is being performed. [Figure 10] It is an explanatory diagram illustrating the image data displayed on the work monitor in the situation where the calibration of the monitoring system is being performed. [Figure 11] It is an explanatory diagram illustrating another image data displayed on the work monitor. [Figure 12] It is an explanatory diagram illustrating in plan view another embodiment of the ship monitoring system of the present invention. [Figure 13] It is an explanatory diagram illustrating the monitoring image data displayed on the monitor for monitoring in FIG. 12. [Figure 14] It is an explanatory diagram illustrating in plan view yet another embodiment of the ship monitoring system of the present invention.

Embodiments for Carrying Out the Invention

[0010] Hereinafter, the ship monitoring system and monitoring method of the present invention will be described based on the embodiments shown in the drawings.

[0011] The ship monitoring system 1 of the present invention illustrated in FIGS. 1 and 2 is a system that monitors ships 20 of a type to be monitored (hereinafter referred to as target ships 20) existing in the surrounding water area SW using a camera device 2 installed at a predetermined monitoring reference position M. This monitoring system 1 is used, for example, when a ship (own ship) 10 that is sailing or stopped monitors a target ship (other ship) 20 existing in the surrounding water area SW, or when monitoring a target ship 20 existing in the surrounding water area SW at a construction site on land facing the water area (for example, a breakwater, a quay wall, a water structure, etc.). In this embodiment, the case where the monitoring system 1 is mounted on the ship 10 and monitors the target ships 20 existing in the surrounding water area SW of the ship 10 is illustrated. In the following description, the ship 10 on which the monitoring system 1 is mounted is referred to as the own ship 10.

[0012] 1 and 2, the surveillance system 1 includes a camera device 2 installed at a predetermined surveillance reference position M, a computing device 3 communicatively connected to the camera device 2, and monitors 4, 6, and 7a communicatively connected to the computing device 3. The surveillance system 1 of this embodiment further includes an input means 5, a management device 7, a positioning means 8, and a warning means 9. The input means 5, the management device 7, the positioning means 8, and the warning means 9 are each communicatively connected to the computing device 3. Each of the above-mentioned communicative connections may be a wired connection or a wireless connection.

[0013] For example, a digital camera or the like is used as the camera device 2. As illustrated in Fig. 1 and Fig. 3, the camera device 2 sequentially acquires image data D1 of the surrounding water area SW of the monitoring reference position M, and sequentially transmits the acquired image data D1 of the surrounding water area SW to the computing device 3. In this embodiment, one camera device 2 is installed on board the ship 10, and the camera device 2 is configured to capture images of the surrounding water area SW in front of the ship 10 (bow side). In Fig. 1, the shooting range (camera angle of view) of the camera device 2 is indicated by a dashed dotted line. In this embodiment, a case is illustrated in which one target ship 20 is present in the shooting range (surrounding water area SW) of the camera device 2.

[0014] When the monitoring system 1 is mounted on the ship 10 to monitor the surrounding water area SW as in this embodiment, the camera device 2 is preferably installed at a relatively high position on the ship or at the end of the ship (for example, the bow, stern, or side) where there are few obstructions in the shooting range. The camera device 2 in this embodiment is configured to be able to adjust the shooting angle in the vertical direction to a desired angle. In this embodiment, the camera device 2 is installed at a position higher than the deck of the ship 10, and the shooting direction of the camera device 2 is set diagonally downward so that a part of the bow of the ship 10 and the surrounding water area SW in front of it are included in the shooting range. The shooting range of the camera device 2 can also be set, for example, to a range in which the ship 10 is not captured. In this embodiment, a case is illustrated in which the horizontal shooting direction of the camera device 2 relative to the ship 10 is fixed, but for example, a rotation mechanism for changing the shooting direction of the camera device 2 can also be provided.

[0015] The monitoring reference position M at which the camera device 2 is installed is not particularly limited as long as it is a position where the surrounding water area SW to be monitored can be photographed, and can be set at any position. When monitoring is performed on the ship 10 as in this embodiment, the monitoring reference position M is set on the ship. When monitoring the surrounding water area SW on land, such as a breakwater, a quay, or an above-water structure, the monitoring reference position M is set on land and the camera device 2 is installed. The photographing range and the number of camera devices 2 to be installed can be appropriately determined depending on the range of the surrounding water area SW to be monitored, etc. As will be exemplified later in another embodiment, the monitoring system 1 of the present invention can also be configured to have multiple camera devices 2.

[0016] A computer or the like is used as the arithmetic unit 3. In this embodiment, the arithmetic unit 3 is disposed in the wheelhouse of the ship 10, but the location of the arithmetic unit 3 is not particularly limited, and the arithmetic unit 3 can be disposed in other locations. The specific configuration of the arithmetic unit 3 will be described later.

[0017] The monitoring system 1 of this embodiment has a work monitor 4 used for advance preparation and checking the operation of the computing device 3 during monitoring, a monitoring monitor 6 mainly used for monitoring the surrounding water area SW, and a management monitor 7a communicably connected to the management device 7. The management device 7 is a computer used by a manager (e.g., another crew member of the ship 10, a manager who manages the navigation of the ship 10 in a remote location, a construction manager, etc.) other than the monitoring worker who uses the monitoring monitor 6 to monitor the surrounding water area SW. The monitoring system 1 only needs to have at least one monitor, and the number of monitors constituting the monitoring system 1 is not particularly limited. The work monitor 4 and the monitoring monitor 6 can be configured as the same monitor, for example. The management device 7 and the management monitor 7a can be provided arbitrarily.

[0018] The input means 5 is a means for inputting information to the arithmetic unit 3. Examples thereof include a keyboard, a mouse, a dial, and a touch panel function provided on the work monitor 4. The positioning means 8 is a means for acquiring the absolute position coordinate information of the monitoring reference position M and the absolute azimuth information of the shooting direction of the camera device 2. For example, a GNSS azimuth meter is used. The positioning means 8 can use, for example, a GNSS receiver as a means for acquiring the absolute position coordinate information of the monitoring reference position M, and a compass such as a gyrocompass or a magnetic compass as a means for acquiring the absolute azimuth information of the shooting direction of the camera device 2.

[0019] In this embodiment, a GNSS azimuth meter is installed on the camera device 2 as the positioning means 8 to acquire the absolute position coordinate information of the monitoring reference position M and the absolute azimuth information of the shooting direction of the camera device 2. The positioning means 8 is not limited to directly acquiring the absolute position coordinate information of the monitoring reference position M and the absolute azimuth information of the shooting direction of the camera device 2. For example, the positioning means 8 can be arranged at a position separated from the monitoring reference position M, and based on the absolute position coordinate information and absolute azimuth information acquired by the positioning means 8 and the relative positional relationship between the monitoring reference position M and the positioning means 8 grasped in advance, the absolute position coordinate information of the monitoring reference position M and the absolute azimuth information of the shooting direction of the camera device 2 can be indirectly acquired.

[0020] The warning means 9 is a means for issuing a warning to the target ship 20 existing in the surrounding water area SW. The warning means 9 can be configured to issue a warning to, for example, the monitoring operator of the own ship 10 performing the monitoring operation or other crew members. In this embodiment, as the warning means 9, an alarm (such as a loudspeaker) that issues a warning by sound is installed on the ship of the own ship 10.

[0021] The arithmetic unit 3 has a memory 3a for storing data and an arithmetic processing unit (CPU) 3b for performing arithmetic processing. In the memory 3a, a detection model for detecting an image of the target ship 20, which is machine-learned using image data of a plurality of types of ships and image data other than ships as teacher data, is stored.

[0022] As shown in Fig. 4, the image data D1 of the surrounding water area SW input from the camera device 2 to the calculation device 3 is sequentially displayed on the work monitor 4 by the calculation processing unit 3b. When the image data D1 of the surrounding water area SW is input, the calculation processing unit 3b is configured to execute a detection process to detect an image of a preset target ship 20 reflected in the image data D1 of the surrounding water area SW based on the input image data D1 of the surrounding water area SW and a detection model pre-stored in the memory 3a, and to display a detection frame (bounding box) F indicating the range of the image of the detected target ship 20 superimposed on the image data D1 of the surrounding water area SW. In Fig. 4, the detection frame F is indicated by a dashed line.

[0023] The teacher data used for machine learning of the detection model includes image data of a plurality of types of ships including the target ship 20 and objects other than ships, each photographed from various angles. The image data of ships, which is the teacher data, is categorized and machine-learned into an image data group of small ships (small vessels such as cargo carriers, fishing boats, and pleasure boats) with a gross tonnage of 499 tons or less that are not required to be equipped with an AIS device, and an image data group of large ships (container ships, tanker ships, etc.) with a gross tonnage of 500 tons or more that are required to be equipped with an AIS device. Examples of image data other than ships include image data of water signs and buoys that may be present in the surrounding waters SW, and heavy machinery and structures present on land around the surrounding waters SW.

[0024] Ships have different shapes and on-board equipment depending on their type and size. In particular, small ships such as cargo handling ships, fishing boats, and pleasure boats with a gross tonnage of 499 tons or less have significantly different shapes and on-board equipment than large ships such as container ships and tankers with a gross tonnage of 500 tons or more. Therefore, it is possible to generate a detection model for detecting the image of the target ship 20 by using image data of multiple types of ships and image data of other than ships as teacher data and having artificial intelligence (AI) learn the differences between ships and other subjects and the characteristics of each type of ship. Examples of machine learning methods include deep learning, neural networks, regression, clustering, and pattern matching.

[0025] As illustrated in Fig. 4, the calculation processing unit 3b further executes a grid display process for displaying a grid G ​​indicating a relative position with respect to the monitoring reference position M superimposed on the water surface in the image data D1 of the surrounding water area SW. As illustrated in Figs. 4 and 5, the grid G ​​is a lattice-like line virtually depicted on the water surface of the surrounding water area SW in a plan view, and is formed of a plurality of vertical lines g1 arranged at a predetermined interval in the horizontal direction (left-right direction) with respect to the monitoring reference position M, and a plurality of horizontal lines g2 arranged at a predetermined interval in the depth direction (front-back direction) with respect to the monitoring reference position M. In the grid display process, the grid G ​​is displayed superimposed as an AR (augmented reality) line on the water surface in the image data D1 of the surrounding water area SW.

[0026] The distance J1 between the vertical lines g1 and the distance J2 between the horizontal lines g2 that make up the grid G ​​can be determined as appropriate, but is set to a predetermined distance of, for example, 10 m or more and 30 m or less. In this embodiment, the distance J1 between the vertical lines g1 and the distance J2 between the horizontal lines g2 are each set to 20 m. For example, the distance J1 between the vertical lines g1 and the distance J2 between the horizontal lines g2 can also be set to different distances. It is preferable to display the grid G ​​in a color (e.g., red, yellow, green, etc.) that is easily distinguishable from the color of the water surface.

[0027] As illustrated in FIG. 4, in this embodiment, the arithmetic processing unit 3b is configured to numerically display, in the grid display process, the longitudinal separation distance from the monitoring reference position M indicated by each horizontal line g2 of the grid G on the image data D1 of the peripheral water area SW. However, this numerical display can be provided arbitrarily. For example, it can also be configured to numerically display the lateral separation distance from the monitoring reference position M indicated by each vertical line g1.

[0028] As illustrated in FIG. 6, the arithmetic processing unit 3b is further configured to sequentially calculate the position information of the target ship 20 based on the positional relationship between the detection frame F and the grid G in the image data D1 of the peripheral water area SW, and execute position information display processing for sequentially displaying the calculated position information of the target ship 20 on the monitoring monitor 6. More specifically, the arithmetic processing unit 3b specifies the relative position of the target ship 20 with respect to the monitoring reference position M based on the positional relationship between the detection frame F and the grid G in the image data D1 of the peripheral water area SW. Next, based on the relative position of the target ship 20 with respect to the specified monitoring reference position M and the position information of the monitoring reference position M, the position information of the target ship 20 is calculated. Then, monitoring image data D2 indicating the calculated position information of the target ship 20 is created, and the created monitoring image data D2 is transmitted to the monitor 6 for display. In this embodiment, the above-described monitoring image data D2 is also transmitted to the management device 7, and the monitoring image data D2 can be displayed on the management monitor 7a.

[0029] To explain the position information display processing in more detail, as illustrated in FIGS. 7 and 8, in the position information display processing, for example, a predetermined position on the outer peripheral edge line of the detection frame F or a predetermined position inside the outer peripheral edge line is set as the detection point P of the target ship 20, and the position information of the target ship 20 is calculated based on the positional relationship between the detection point P and the grid G. FIG. 7 shows an example of the detection frame F and the detection points P (P1, P2) when the bow direction of the target ship 20 shown in the image data D1 of the peripheral water area SW is facing forward. FIG. 8 shows an example of the detection frame F and the detection points P (P1 to P4) when the bow direction of the target ship 20 shown in the image data D1 of the peripheral water area SW is facing right.

[0030] The detection model that detects the image of the target ship 20 uses not only the hull of the target ship 20 reflected in the image data D1 of the surrounding waters SW but also the water surface and waves reflected around the hull as discrimination elements to determine whether or not the image is of the target ship 20. For this reason, as exemplified in Figures 7 and 8, the detection frame F displayed by the detection model is slightly larger than the image of the target ship 20.

[0031] The detection point P1 shown in FIG. 7 and FIG. 8 illustrates a case where a predetermined position on the bottom line of the detection frame F is set as the detection point P of the target ship 20. When the detection point P1 is set in this manner, the position information of the target ship 20 is calculated based on a position in the detection frame F that is close to the monitoring reference position M, regardless of the bow direction of the target ship 20, so that monitoring can be performed on a safer side than when the detection point P is set at a predetermined position above the bottom line of the detection frame F. More preferably, a predetermined position in the center of the bottom line of the detection frame F (specifically, for example, a range in which the distance in the left-right direction from the center point of the bottom line is 20% or less of the length XC of the bottom line) or even more preferably the center point of the bottom line of the detection frame F is set as the detection point P1 of the target ship 20. Since the image of the target ship 20 is displayed approximately in the center of the detection frame F regardless of the bow direction of the target ship 20, setting the detection point P1 at the above-mentioned position is advantageous for calculating the position information of the target ship 20 with higher accuracy.

[0032] 7 and 8 illustrate an example in which a predetermined position inside the outer perimeter line of detection frame F is set as the detection point P of the target ship 20. Since the image of the target ship 20 is displayed inside the outer perimeter line of detection frame F, setting detection point P2 in this manner makes it possible to calculate position information of the target ship 20 with relatively high accuracy regardless of the bow direction of the target ship 20. More preferably, detection point P2 should be set at a predetermined position a predetermined distance Y1 above the bottom line of detection frame F, even more preferably at a predetermined position a predetermined distance Y1 above the center of the bottom line of detection frame F, and even more preferably at a predetermined distance Y1 above the center point of the bottom line of detection frame F.

[0033] The above-mentioned predetermined distance Y1 is preferably set under the condition that the detection point P2 is located at the bottom edge of the image of the target ship 20. The predetermined distance Y1 is a relative distance with respect to the size of the detection frame F, and the predetermined distance Y1 is set, for example, to a distance of 5% to 30% of the height YC of the detection frame F, more preferably a distance of 10% to 20%. By setting the detection point P2 in this manner, regardless of the bow direction of the target ship 20, the position information of the target ship 20 can be calculated more accurately based on a position that is relatively close to the monitoring reference position M, and monitoring can be performed on the safe side. When the calculation processing unit 3b does not detect the bow direction of the target ship 20 in the image data D1 of the surrounding waters SW, it is preferable to set the detection point P at the position of the above-mentioned detection point P1 or detection point P2.

[0034] For example, in the detection process for detecting image data of the target ship 20 reflected in the image data D1 of the surrounding water area SW described above, the arithmetic processing unit 3b may further be configured to detect the bow direction of the target ship 20 in the image data D1 of the surrounding water area SW from the image data of the target ship 20 reflected in the image data D1 of the surrounding water area SW. Specifically, for example, the arithmetic processing unit 3b may be configured to detect the general bow direction of the target ship 20 by determining, using a detection model, which category the target ship 20 reflected in the image data D1 of the surrounding water area SW belongs to: front, rear, rightward, or leftward. The detection model for detecting the bow direction of the target ship 20 may be generated by subjecting image data of multiple types of ships photographed from various directions to categorization by ship direction and machine learning.

[0035] The bow direction of the target ship 20 in the image data D1 of the surrounding waters SW can be detected, for example, by the calculation processing unit 3b continuously calculating the position information of the target ship 20 and grasping the traveling direction of the target ship 20. In this case, in generating the detection model, it becomes possible to more easily detect the bow direction of the target ship 20 without having to categorize image data of multiple types of ships photographed from various directions by ship direction and perform machine learning.

[0036] As described above, in the case where the calculation processing unit 3b is configured to detect the bow direction of the target ship 20 in the image data D1 of the surrounding waters SW, the position of the detection point P with respect to the detection frame F can be set under different conditions depending on, for example, the bow direction of the target ship 20, whether the bow direction of the target ship 20 is forward or backward, or whether the bow direction is to the right or left, depending on the bow direction of the target ship 20. As illustrated in Fig. 7, when the bow direction of the target ship 20 is forward or backward, the bow or stern of the target ship 20 is reflected approximately in the center of the detection frame F in the left-right direction, so it is advisable to set the detection point P at a predetermined position at the center or center point of the bottom line of the detection frame F, such as the detection point P1 described above, or at a predetermined position a predetermined distance Y1 above the center or center point of the bottom line of the detection frame F, such as the detection point P2 described above.

[0037] 8, when the bow of the target ship 20 faces right, the bow of the target ship 20 will be reflected on the right side of the detection frame F, so for example, the lower right corner of the detection frame F should be set as the detection point P3. When the target ship 20 faces left, the bow of the target ship 20 will be reflected on the left side of the detection frame F, so for example, the lower left corner of the detection frame F should be set as the detection point P3. Setting the detection point P3 in this manner makes it possible to more accurately calculate position information on the bow side of the target ship 20, and to perform monitoring from a safer side, even when the bow of the target ship 20 faces sideways.

[0038] More preferably, when the bow direction of the target ship 20 faces rightward, detection point P4 should be set at a predetermined position a predetermined distance X1 to the left of the right edge line of detection frame F and a predetermined distance Y1 above the bottom edge line of detection frame F. When the bow direction of the target ship 20 faces leftward, detection point P4 should be set at a predetermined position a predetermined distance X1 to the right of the left edge line of detection frame F and a predetermined distance Y1 above the bottom edge line of detection frame F. The aforementioned predetermined distance X1 should be set under the condition that detection point P4 is located at the end position of the bow side of the target ship 20 in the left-right direction.

[0039] The above-mentioned predetermined distance X1 is a relative distance with respect to the size of the detection frame F, and the predetermined distance X1 is set, for example, to a distance of 1% to 15% of the left-right width XC of the detection frame F, and more preferably a distance of 3% to 10%. By setting the detection point P4 in this manner, when the bow of the target ship 20 is facing sideways, it is possible to monitor the safe side while calculating position information of the bow side of the target ship 20 with even greater accuracy. The above-mentioned predetermined distance Y1 when the bow of the target ship 20 is facing forward or backward and the above-mentioned predetermined distance Y1 when the bow of the target ship 20 is facing right or left can also be set to different distances.

[0040] For example, if a monitoring operator checks the positional relationship between the image of the target ship 20 reflected in the image data of the surrounding waters SW displayed on the monitoring monitor 6 as advance preparation and the detection frame F, and the monitoring operator uses the input means 5 to set the position of the detection point P relative to the detection frame F, the position of the detection point P can be easily set. Since the positional relationship between the image of the target ship 20 and the detection frame F is roughly the same even when the size of the target ship 20 or the distance of the target ship 20 from the monitoring reference position M is different, once the monitoring operator has set the detection point P as described above, that setting can be used continuously.

[0041] As shown in Fig. 6, in this embodiment, the calculation processing unit 3b is configured to display a plan view showing the planar position of the target ship 20 in the surrounding water area SW on the monitoring monitor 6 as monitoring image data D2 showing the position information of the target ship 20 by the above-mentioned position information display process. The position information of the target ship 20 can also be displayed on the management monitor 7a. In Fig. 6, an icon showing the position of the ship 10 and an icon showing the position of the target ship 20 present in the surrounding water area SW are each displayed on the plan view. In this embodiment, an arrow is displayed as the icon of the target ship 20, and the bow direction of the target ship 20 is displayed in the direction of the arrow.

[0042] The bow direction of the target ship 20 can be identified by continuously calculating the position information of the target ship 20 by the calculation processing unit 3b and grasping the traveling direction of the target ship 20. For example, if the calculation processing unit 3b is configured to detect the bow direction of the target ship 20 in the image data D1 of the surrounding water area SW from the image data of the target ship 20 reflected in the image data D1 of the surrounding water area SW, the bow direction of the target ship 20 can be identified from the detection result. The shapes and designs of the icons representing the own ship 10 and the target ship 20 are not limited to this embodiment, and various other configurations are possible. For example, it is also possible to configure the monitoring image data D2 to display only the range of the surrounding water area SW without displaying the position of the own ship 10.

[0043] In this embodiment, the calculation processing unit 3b is further configured to display, in the position information display process, a coordinate display W1 indicating, in numerical form, the relative position coordinates of the target ship 20 with respect to the own ship 10 on the monitoring image data D2 as the position information of the target ship 20. In this embodiment, the coordinate display W1 is configured to display, as the above-mentioned relative position coordinates, the left-right relative position coordinates and the fore-aft relative position coordinates of the target ship 20 with respect to the monitoring reference position M when the monitoring reference position M is set as the coordinate reference point (X: 0m, Y: 0m), and the distance between the monitoring reference position M and the target ship 20 (detection point P).

[0044] Although not shown in Fig. 6, in this embodiment, the calculation processing unit 3b is further configured to calculate the absolute position coordinates of the target ship 20 in the position information display process based on the absolute position coordinate information of the monitoring reference position M input from the positioning means 8, the absolute orientation information of the shooting direction of the camera device 2, and the relative position of the target ship 20 with respect to the identified monitoring reference position M, and to display the absolute position coordinates (latitude, longitude) of the target ship 20 (detection point P) in the monitoring image data D2 as the position information of the target ship 20. Although not shown in Fig. 6, for example, a grid G ​​in plan view can be displayed on a plan view showing the position information of the target ship 20.

[0045] The calculation processing unit 3b in this embodiment is further configured to execute a warning determination process that issues an activation command to the warning means 9 when it determines that the distance between the ship 10 and the target ship 20 is shorter than a preset warning distance based on the position information of the target ship 20 calculated by the position information display process and the warning distance previously set for the ship 10. The warning distance can be set as appropriate. In this embodiment, when the calculation processing unit 3b issues an activation command to an alarm device, which is the warning means 9, the alarm device is configured to emit a warning sound.

[0046] Furthermore, in this embodiment, as illustrated in Fig. 6, when it is determined that the distance between the ship 10 and the target ship 20 is shorter than a preset warning distance, a warning display W2 is displayed for the target ship 20 in the monitoring image data D2. That is, in this embodiment, the monitors 6, 7a are also configured to function as the warning means 9. Fig. 6 illustrates an example in which the warning display W2 for the target ship 20 is displayed as a dashed rectangular frame surrounding the icon of the target ship 20, but the manner in which the warning display W2 is displayed is not particularly limited, and for example, the warning display W2 may be configured to display text information or a mark.

[0047] As described above, the memory 3a of the calculation device 3 stores (contains) computer programs for executing the above-mentioned detection processing, grid display processing, position information display processing, and warning determination processing, and when image data D1 of the surrounding water area SW is input to the calculation device 3, the calculation processing unit 3b is configured to repeatedly execute the detection processing, grid display processing, position information display processing, and warning determination processing for each shooting cycle of the camera device 2 based on the respective computer programs stored in the memory 3a.

[0048] As a result, the work monitor 4 displays image data in which a detection frame F and a grid G ​​indicating the range of the image of the target ship 20 are superimposed on the image data D1 of the surrounding waters SW as a moving image with little time lag and in near real time. The monitoring monitor 6 and the management monitor 7a are configured to display monitoring image data D2 indicating the position information of the target ship 20 present in the surrounding waters SW as a moving image with little time lag and in near real time. When multiple target ships 20 are shown in the image data D1 of the surrounding waters SW, the calculation processing unit 3b is configured to execute detection processing, position information display processing, and warning determination processing for each target ship 20.

[0049] Next, a method for monitoring a ship using this monitoring system 1 will be described.

[0050] As a preparation before monitoring by the monitoring system 1, a detection model for detecting an image of a target ship 20 that has been machine-learned using image data of multiple types of ships and image data of non-ships as teacher data is generated, and the detection model is stored in the memory 3a of the computing device 3. As a preparation for grid display processing by the computation processing unit 3b of the computing device 3, calibration is performed to align the position of the grid G ​​with the water surface in the image data D1 of the surrounding water area acquired by the camera device 2 and adjust the size of the grid G. Before performing calibration of the grid display processing, the height H from the water surface position WL of the surrounding water area SW to the shooting position of the camera device 2 (monitoring reference position M) when the camera device 2 is installed at the monitoring reference position M, and the shooting range of the camera device 2 (camera angle and camera magnification) are determined in advance.

[0051] Calibration of the grid display process can be performed on land or on water. As illustrated in FIG. 9, this embodiment illustrates a case where calibration is performed on land. In calibration on land, the ground is regarded as the water surface, and the height H from the water surface position WL when the camera device 2 is installed at the monitoring reference position M and the shooting range of the camera device 2 are reproduced on land. Then, a target 30 such as a road cone is placed on the ground within the shooting range of the camera device 2, and the distance R in the front-rear direction and the distance in the left-right direction from the shooting position of the camera device 2 to the target 30 are actually measured. In this embodiment, the distance R in the front-rear direction from the shooting position of the camera device 2 to the target 30 is set to 80 m, and the distance in the left-right direction is set to 0 m.

[0052] 10, the ground and the target 30 are photographed by the camera device 2, and while the image data acquired by the camera device 2 is being checked on the work monitor 4, the display position of the grid G ​​relative to the image data is adjusted by regarding the ground reflected in the image data as the water surface. Furthermore, the display position and display size of the grid G ​​relative to the image data are adjusted using the position of the target 30 reflected in the image data as a guide. In this embodiment, since the target 30 is located 80 m forward from the shooting position of the camera device 2, the display position and display size of the grid G ​​are adjusted by the input means 5 so that the horizontal line g2 of the grid G ​​indicating a position 80 m forward from the shooting position matches the position of the target 30 on the image data. For example, if the target 30 is located 70 m forward from the shooting position of the camera device 2, the display position and display size of the grid G ​​are adjusted by the input means 5 so that the position of the target 30 matches a position on the image data in the grid G ​​indicating a position 70 m forward from the shooting position (in this embodiment, a position between the horizontal line g2 of the grid G ​​indicating a position 60 m forward from the shooting position and the horizontal line g2 of the grid G ​​indicating a position 80 m forward from the shooting position).

[0053] In the case where the camera device 2 is configured to be changeable in terms of the shooting direction and zoom magnification, the shooting conditions of the camera device 2, such as the shooting direction and zoom magnification, are changed to similarly adjust the display position and display size of the grid G ​​on the image data. Then, data on the correspondence between the shooting conditions of the camera device 2 (shooting direction and zoom magnification) and the display position and display size of the grid G ​​is acquired in advance, and the correspondence data is included in the programming of the grid display process. In this way, even if the shooting conditions of the camera device 2 are changed, it becomes possible to execute grid display process that automatically adjusts the display position and display size of the grid G ​​to match the shooting conditions.

[0054] When calibrating the grid display process on water, with the camera device 2 installed at the monitoring reference position M of the ship 10, a target 30 such as a buoy is placed on the water surface within the range photographed by the camera device 2, and the longitudinal separation distance R and lateral separation distance from the photographing position of the camera device 2 to the target 30 are actually measured. Next, the water surface and the target 30 are photographed by the camera device 2, and while checking the image data displayed on the work monitor 4, the display position of the grid G ​​is adjusted to match the water surface reflected in the image data. Furthermore, the display position and display size of the grid G ​​relative to the image data are adjusted using the position of the target 30 reflected in the image data as a guide.

[0055] For example, the grid display process can be calibrated using a ship parked on the water surface within the range of shooting by the camera device 2 as the target 30. In this case, the absolute position coordinate information acquired by a GNSS compass or GNSS receiver mounted on the ship serving as the target 30 can be used to easily determine the longitudinal separation distance R and the lateral separation distance from the shooting position of the camera device 2 to the ship serving as the target 30. Calibration of the grid display process can be easily performed on the water surface, but since the target 30 placed on the water surface is subject to swaying due to the effects of waves, etc., it is more advantageous to perform the calibration on land in order to improve the accuracy of the calibration.

[0056] When performing monitoring work using the monitoring system 1, a target ship 20 (type of ship to be monitored) is set in the calculation device 3. In this embodiment, ships with a gross tonnage of 499 tons or less are set as the target ship 20. The target ship 20 can be configured to be set in advance in the calculation device 3, or the monitoring operator can use the input means 5 to select the type of ship to be monitored from a plurality of options provided in advance. For example, all types of ships can be set as the target ship 20.

[0057] In monitoring operations using the monitoring system 1, the camera device 2 sequentially acquires image data D1 of the surrounding water area SW of the monitoring reference position M, and the acquired image data D1 of the surrounding water area SW is sequentially input to the calculation device 3. In this embodiment, one camera device 2 sequentially acquires image data D1 of the surrounding water area SW ahead of the ship 10, and the acquired image data D1 of the surrounding water area SW is sequentially input to the calculation device 3.

[0058] Then, using the calculation device 3, based on the detection model stored in the memory 3a and the image data D1 of the surrounding water area SW input from the camera device 2, the calculation processing unit 3b detects an image of a predetermined target ship 20 that is reflected in the image data D1 of the surrounding water area SW, and executes a detection process in which a detection frame F indicating the range of the image of the detected target ship 20 is superimposed on the image data D1 of the surrounding water area SW.

[0059] Next, the calculation processing unit 3b executes a grid display process in which a grid G ​​indicating a relative position to the monitoring reference position M is superimposed on the water surface in the image data D1 of the surrounding water area SW. After that, the calculation processing unit 3b executes a position information display process in which the calculation information of the target ship 20 is calculated based on the positional relationship between the detection frame F and the grid G ​​in the image data D1 of the surrounding water area SW, and the calculated position information of the target ship 20 is displayed on the monitors 6 and 7a. The monitoring operator monitors the surrounding water area SW of the ship 10 by checking the position information of the target ship 20 displayed on the monitoring monitor 6 and the image data D1 of the surrounding water area SW displayed on the work monitor 4. The manager can check the situation of the surrounding water area SW of the ship 10 by checking the position information of the target ship 20 displayed on the management monitor 7a.

[0060] In this embodiment, the calculation processing unit 3b executes a warning determination process to issue an activation command to the warning means 9 when it determines that the separation distance between the ship 10 and the target ship 20 is shorter than the preset warning distance based on the position information of the target ship 20 calculated by the position information display process and the warning distance preset for the ship 10. When the warning means 9 issues a warning, the crew of the ship 10 appropriately performs visual monitoring of the target ship 20 and evasive action to avoid a collision between the ship 10 and the target ship 20. The warning by the warning means 9 continues until the target ship 20 moves to a position farther away than the warning distance, unless the crew of the ship 10 intentionally operates it to stop.

[0061] In this way, in the present invention, the camera device 2 installed at a predetermined monitoring reference position M sequentially acquires image data D1 of the surrounding water area SW of the monitoring reference position M, and the acquired image data D1 of the surrounding water area SW is sequentially input to the calculation device 3. Then, a detection model for detecting an image of a target ship 20 that has been machine-learned using image data of multiple types of ships and image data of non-ships as teacher data is stored in the memory 3a of the calculation device 3, so that the calculation processing unit 3b can detect an image of a preset target ship 20 reflected in the image data D1 of the surrounding water area SW based on the detection model and the image data D1 of the surrounding water area SW, and display a detection frame F indicating the range of the image of the detected target ship 20 superimposed on the image data D1 of the surrounding water area SW.

[0062] Furthermore, by displaying a grid G ​​indicating a relative position to the monitoring reference position M on the water surface in the image data D1 of the surrounding water area SW, the position information of the target ship 20 can be calculated based on the positional relationship between the detection frame F and the grid G ​​in the image data D1 of the surrounding water area SW. Then, by displaying the calculated position information of the target ship 20 on the monitors 6 and 7a, it becomes possible to grasp the position information of the target ship 20 existing in the water area to be monitored (surrounding water area SW) more reliably with less time lag and in near real time. As a result, in addition to visual monitoring by the monitoring operator, the safety of the surrounding water area SW of the monitoring reference position can be more reliably ensured. There is also the advantage that the effort required for visual monitoring can be significantly reduced.

[0063] In the present invention, by using image data D1 of the surrounding waters SW acquired by the camera device 2, even small ships that are difficult to detect by a radar device and exist within a short distance range of the monitoring reference position M can be reliably detected almost in real time with little time lag. Since small ships such as fishing boats and pleasure boats that are not required to be equipped with AIS can move in an unpredictable manner, it is very useful to set ships with a gross tonnage of 499 tons or less as the target ships 20.

[0064] In this embodiment, by configuring the position information display process by the calculation processing unit 3b to display on the monitors 6, 7a a plan view showing the planar position of the target ship 20 in the surrounding water area SW as position information of the target ship 20, it becomes very easy to grasp the position of the target ship 20 in the surrounding water area SW, and also makes it easier to grasp the relative positional relationship between the own ship 10 (monitoring reference position M) and the target ship 20.

[0065] As in this embodiment, a positioning means 8 is provided, and the absolute position coordinates of the target ship 20 are calculated in a position information display process by the calculation processing unit 3b based on the absolute position coordinate information of the monitoring reference position M acquired by the positioning means 8, the absolute orientation information of the shooting direction of the camera device 2, and the relative position of the target ship 20 with respect to the monitoring reference position M. If the absolute position coordinates of the target ship 20 are displayed on the monitors 6, 7a as the position information of the target ship 20, the absolute coordinate position of the target ship 20 can be more clearly grasped.

[0066] In the present invention, the position information of the target ship 20 is not limited to being displayed on the monitors 6, 7a as a plan view showing the planar position of the target ship 20, but for example, the position information of the target ship 20 can be configured to be displayed on the monitors 6, 7a as numerical values ​​of the relative position coordinates of the target ship 20 with respect to the monitoring reference position M or numerical values ​​of the absolute position coordinates of the target ship 20. In other words, the display form of the position information of the target ship 20 is not limited to the configuration of this embodiment, and various other configurations are possible.

[0067] If the calculation processing unit 3b is configured to execute a warning determination process that issues an activation command to the warning means 9 when it determines that the distance between the ship 10 and the target ship 20 is shorter than a preset warning target distance, the warning means 9 can quickly issue a warning to the target ship 20 whose distance from the ship 10 is shorter than the warning target distance, which is advantageous for avoiding a collision between the ship 10 and the target ship 20. Furthermore, the issuance of a warning from the warning means 9 can enable the monitoring operator to more reliably recognize the target ship 20 that is highly in need of monitoring.

[0068] The warning means 9 can be configured as, in addition to an alarm device, a warning light or an electronic bulletin board that visually displays a warning, a warning device (such as an alarm) or a warning light that issues a warning to the crew of the ship 10, a portable terminal, or the like. The portable terminal is a terminal carried by the crew on board the ship 10, and specifically includes, for example, a wristwatch, a mobile phone, a portable tablet, and the like that have a vibration function or a text display function. If the warning means 9 is configured to issue a warning to the monitoring operator of the ship 10 or other crew members, even if the monitoring operator misses the target ship 20 during visual monitoring, the presence of the target ship 20 can be recognized. This is therefore more advantageous in ensuring the safety of the water area SW surrounding the ship 10. In the monitoring system 1 of the present invention, the warning determination process by the warning means 9 and the calculation device 3 is not an essential configuration, and the warning determination process and the warning by the warning means 9 may not be performed.

[0069] Although not provided in this embodiment, for example, a lighting device such as a searchlight can be attached to the camera device 2, and the lighting device can be configured to irradiate light onto the shooting range of the camera device 2. For example, an illuminance sensor that detects the brightness outside the ship can be provided, and when the illuminance of the surrounding water area SW is equal to or higher than a set value, the lighting device can be configured to go into standby mode, and when the illuminance of the surrounding water area SW falls below the set value, the lighting device can be automatically started to irradiate light.

[0070] For example, an infrared night vision camera or a low-illumination camera (high-sensitivity camera) capable of taking pictures even in dark places can be used as the camera device 2. A low-illumination camera (high-sensitivity camera) is equipped with an image processing function such as a wide dynamic range function (WDR function) that enables taking pictures even in dark places, and is a camera that can take pictures even in low-illumination surrounding water areas SW, where the illuminance is about 0.005 Lux. When lighting equipment is provided or an infrared night vision camera or a low-illumination camera is used as the camera device 2, it becomes possible to carry out monitoring work even at night when the surrounding water areas SW are dark or in bad weather.

[0071] The monitoring system 1 of the present invention can be configured, for example, to have the monitoring operator check screen data D1 of the surrounding water area SW displayed on the monitor 4 to see whether or not a subject determined to be a target ship 20 in the detection process is actually a target ship 20, and to have the monitoring operator store in the memory 3a, via the input means 5, the result of the determination of the detection process of the target ship 20 by the arithmetic processing unit as to whether or not the subject is reflected in the image data of the surrounding water area SW. With this configuration, the amount of training data for the detection model can be increased even while the monitoring system 1 is in use, and the detection model can be improved by machine learning, thereby improving the accuracy of the detection process.

[0072] 11, in the monitoring system 1 of the present invention, for example, a detection model stored in the memory 3a of the computing device 3 is machine-trained by adding image data of a ship 21 excluded from monitoring (hereinafter referred to as an excluded ship 21) as teacher data, and when the detection model detects a preset excluded ship 21 reflected in image data D1 of the surrounding water area SW, a detection frame F is intentionally not displayed for the image of the detected excluded ship 21. Also, for example, a configuration can be adopted in which the computing processing unit 3b does not calculate position information of the excluded ship 21, and the position information of the excluded ship 21 is not intentionally displayed on the monitors 6, 7a.

[0073] Examples of the excluded vessel 21 include work vessels and soil transport vessels that perform construction work together with the vessel 10. Work vessels and soil transport vessels that perform construction work together with the vessel 10 are in constant close contact with the vessel 10, so there is an extremely low need to monitor them. Since the excluded vessels 21 can be identified in advance, abundant image data of the excluded vessel 21 can be prepared as training data. By adding image data of the excluded vessel 21 as training data in the machine learning of the detection model, the probability of erroneously detecting the excluded vessel 21 as the target vessel 20 in the detection process using the detection model can be reduced, which is advantageous for improving the accuracy of the detection process.

[0074] In particular, monitoring vessels and work vessels that carry out construction work together with the ship 10 frequently travel through the surrounding waters SW of the ship 10. This makes it possible to more reliably avoid frequent display of position information of excluded vessels 21 that do not require monitoring on the monitors 6, 7a, and frequent and continuous issuance of warnings to excluded vessels 21 that do not require warnings. As a result, the crew of the ship 10 can concentrate on monitoring the target vessel 20 that requires monitoring more, even when, for example, the target vessel 20 and the excluded vessel 21 are both present in the surrounding waters SW.

[0075] As exemplified in FIG. 11, the monitoring system 1 of the present invention can also be configured to display a warning display W2 indicating the position of the target ship 20 on the image data D1 of the surrounding water area SW displayed on the work monitor 4 as a warning by the warning means 9. In this embodiment, a dashed rectangular frame is displayed outside the detection frame F as the warning display W2. It is preferable that the warning display W2 be displayed in a manner that can be clearly distinguished from the detection frame F. Such a configuration allows the monitoring operator to more reliably recognize the position and direction of the target ship 20 that is the subject of the warning. The warning display W2 may be displayed in a manner different from that of this embodiment as long as it indicates the position of the target ship 20 to the crew.

[0076] 12 and 13 illustrate another embodiment of the surveillance system 1 of the present invention.

[0077] The monitoring system 1 of this embodiment has a plurality of camera devices 2. As illustrated in Fig. 12, in this embodiment, one camera device 2 that photographs the front of the hull, two camera devices 2 that photograph the port and starboard sides of the hull, and one camera device 2 that photographs the rear of the hull are installed on the ship 10. In this embodiment, image data D1 of the surrounding water area SW acquired by each camera device 2 is input to the calculation device 3 sequentially.

[0078] The arithmetic processing unit 3b of the arithmetic device 3 is configured to execute a detection process, a grid display process, a position information display process, and a warning determination process for the image data D1 of the surrounding water area SW input from each camera device 2. As illustrated in FIG. 13, in the position information display process, the monitors 6 and 7a are configured to display monitoring image data D2 that integrates the position information of the target ship 20 in the image data D1 of the surrounding water area SW input from each camera device 2. In this embodiment, a grid G ​​is superimposed on a plan view of the monitoring image data D2. By superimposing the grid G ​​on the plan view in this way, the position of the target ship 20 in the surrounding water area SW can be more easily grasped.

[0079] FIG. 14 illustrates yet another embodiment of the monitoring system 1 of the present invention.

[0080] 14, the monitoring system 1 of the present invention can also be configured to monitor a target ship 20 that exists in a surrounding water area SW facing the land L by installing a camera device 2 on land L facing a water area such as a breakwater, a quay, or an above-water structure. In this embodiment, two camera devices 2 with different shooting directions are installed on the land L, and image data D1 of the surrounding water area SW acquired by each camera device 2 is sequentially input to the calculation device 3.

[0081] The arithmetic processing unit 3b of the arithmetic device 3 is configured to execute detection processing, grid display processing, position information display processing, and warning determination processing for the image data D1 of the surrounding water area SW input from each of the camera devices 2. In the position information display processing, the monitors 6, 7a are configured to display monitoring image data D2 that integrates the position information of the target ship 20 in the image data D1 of the surrounding water area SW input from each of the camera devices 2. As in this embodiment, the monitoring system 1 of the present invention can also be used for monitoring the surrounding water area SW facing the land L, and is therefore highly versatile.

[0082] In addition to the embodiments exemplified above, the monitoring system 1 of the present invention can also be configured such that, for example, when monitoring the surrounding waters SW of a ship 10 moored near land L such as a quay, camera devices 2 for acquiring image data of the surrounding waters SW are installed on the ship 10 and on land L. [Explanation of symbols]

[0083] 1. Surveillance System 2 Camera equipment 3 Computing device 3a Memory 3b Processing section 4 (working) monitors 5. Input methods 6 Monitor 7 Management device 7a (Administrative) Monitor 8 Positioning Methods 9 Warning measures 10 Ship (own ship) 20 Types of ships monitored (target ships) 21 Ships not subject to monitoring (exempt ships) 30 Target D1 Image data of surrounding waters D2 Surveillance image data M Monitoring reference position G Grid g1 (grid) vertical line g2 (grid) horizontal line F Detection frame P, P1~P4 detection points SW Surrounding waters L Athletics W1 coordinate display W2 Warning display

Claims

1. The system comprises a camera device that is installed at a predetermined monitoring reference position and sequentially acquires image data of the surrounding water area of ​​the monitoring reference position, a calculation device to which the image data of the surrounding water area acquired by the camera device is sequentially input, and a monitor that is communicatively connected to the calculation device, The arithmetic device has a memory in which a detection model for detecting images of a ship of a monitoring target type that has been machine-learned using image data of a plurality of types of ships and image data of non-ships as training data is stored, and a calculation processing unit that performs calculation processing; The ship monitoring system is characterized in that the calculation processing unit is configured to execute a detection process that detects images of the predetermined ship type to be monitored that are reflected in the image data of the surrounding water area based on the detection model and the image data of the surrounding water area input from the camera device, and superimposes a detection frame indicating the range of the detected image of the ship of the monitored type on the image data of the surrounding water area, a grid display process that superimposes a grid indicating a relative position to the monitoring reference position on the water surface in the image data of the surrounding water area, and a position information display process that calculates position information of the ship of the monitored type based on the positional relationship between the detection frame and the grid in the image data of the surrounding water area, and displays the calculated position information of the ship of the monitored type on the monitor.

2. A ship monitoring system as described in claim 1, wherein the position information display process is configured to display on the monitor a plan view showing the planar position of the monitored type of ship in the surrounding waters as position information of the monitored type of ship.

3. A ship monitoring system as described in claim 1 or 2, further comprising a positioning means for acquiring absolute position coordinate information of the monitoring reference position and absolute orientation information of the shooting direction of the camera device, and in the position information display process, absolute position coordinates of the ship to be monitored are calculated based on the absolute position coordinate information of the monitoring reference position and the absolute orientation information of the shooting direction of the camera device acquired by the positioning means, and the relative position of the ship to be monitored relative to the monitoring reference position, and the absolute position coordinates of the ship to be monitored on the monitor as position information of the ship to be monitored.

4. A ship monitoring system as described in claim 1 or 2, wherein the position information display process sets a predetermined position on the bottom line of the detection frame as a detection point for the ship of the monitored type, and calculates position information for the ship of the monitored type based on the positional relationship between the detection point and the grid.

5. A ship monitoring system as described in claim 1 or 2, wherein the position information display process sets a predetermined position inside the outer peripheral line of the detection frame as a detection point for the ship of the monitored type, and calculates position information for the ship of the monitored type based on the positional relationship between the detection point and the grid.

6. a detection model for detecting images of a ship of a predetermined type that is a target for monitoring that is machine-learned using image data of a plurality of types of ships and image data of items other than ships as training data, and a detection frame indicating the range of the detected images of ships of the target type that are reflected in the image data of the surrounding waters based on the detection model and the image data of the surrounding waters input from the camera device, and a detection frame indicating the range of the detected images of ships of the target type that are a target for monitoring that are superimposed on the image data of the surrounding waters, a grid indicating a relative position with respect to the monitoring reference position is superimposed on the water surface in the image data of the surrounding waters, a positional information of the ship of the target type that is a target for monitoring ...

Images