Ship recognition device, ship recognition method, and ship recognition program

Abstract

A ship recognition device according to an embodiment of the present invention includes an image acquisition unit that acquires images captured from the sky, a contour extraction unit that extracts, on the basis of the image, the contour of the ship including a wake thereof, a convex hull creation unit that obtains, on the basis of the image, the convex hull of the ship including a wake thereof, and a movement recognition unit that recognizes the ship on the basis of the contour and the convex hull.

Description

Ship recognition device, ship recognition method, and ship recognition program

[0001] The present invention relates to a ship recognition device, a ship recognition method, and a ship recognition program.

[0002] In recent years, due to the increasing attention to the ocean, which can affect national-level issues such as national defense and ensuring the supply chain, and the growing demand for autonomous operation of ships, Maritime Domain Awareness (MDA) has become extremely important. Maritime Domain Awareness is an effort to effectively and efficiently grasp the situation of the ocean by aggregating and sharing various information related to the ocean.

[0003] For example, in order to appropriately take defensive measures such as early detection of ships that violate territorial waters or to improve the safety of maritime traffic, it is necessary to recognize the movements of ships from images taken from above by artificial satellites, aircraft, helicopters, drones, etc.

[0004] Note that Patent Document 1 describes extracting regions where the luminance changes rapidly as edges in order to detect wake waves or moving objects from image data. Patent Document 2 describes estimating wake waves based on Automatic Identification System (AIS) information transmitted by an AIS installed on other ships and graphically displaying the wake waves on a display unit. Patent Document 3 describes observing the water flow in a predetermined water area based on an image taken by a thermal camera. Patent Document 4 describes a method for detecting a bow wave velocity vector.

[0005] Japanese Patent No. 7010422 JP-A-2016-55772 JP-A-2015-127669 WO 2014 / 192530 pamphlet

[0006] The problem to be solved by the present invention is to recognize ships from captured images. However, it is not limited to this purpose, and problems corresponding to the respective effects according to the configurations of the respective embodiments described later may also be regarded as the problems of the present invention.

[0007] A ship recognition device according to one aspect of the present invention comprises: an image acquisition unit that acquires an image taken from above; a contour extraction unit that extracts the outline of a ship, including its wake, based on the image; a convex hull creation unit that determines the convex hull of the ship, including its wake, based on the image; and a recognition unit that recognizes the ship based on the outline and the convex hull.

[0008] Furthermore, in the ship recognition device, the recognition unit may recognize the movement of the ship based on a first area of ​​the contour region enclosed by the contour and a second area of ​​the convex hull region enclosed by the convex hull.

[0009] Furthermore, in the ship recognition device, the recognition unit may recognize that the ship is moving if the value obtained by dividing the first area by the second area is less than a threshold, and recognize that the ship is stopped otherwise.

[0010] Furthermore, in the ship recognition device, the recognition unit may recognize that the ship is moving if the value obtained by subtracting the first area from the second area is greater than a threshold, and recognize that the ship is stopped otherwise.

[0011] Furthermore, the ship recognition device may include a ship detection unit that detects ships in the acquired image, and when the ship detection unit detects a ship in the image, the contour extraction unit extracts the contour and the convex hull creation unit calculates the convex hull.

[0012] Furthermore, in the ship recognition device, the ship detection unit may process the image based on information regarding weather and / or sea conditions in the region corresponding to the image.

[0013] Furthermore, in the ship recognition device, the recognition unit may estimate the speed of the ship based on the contour and the convex hull.

[0014] Furthermore, in the ship recognition device, the recognition unit may divide the convex hull region surrounded by the convex hull into a plurality of divided regions along its short axis, and recognize the front and rear of the ship based on the first and second divided regions, which are closest to the ends of the convex hull region in the long axis direction.

[0015] The recognition unit may calculate the area ratio for the first and second divided regions by the area of ​​the convex hull region, respectively, and if the area ratio of the first divided region is greater than the area ratio of the second divided region, it may recognize the first divided region as the front region and the second divided region as the back region.

[0016] The recognition unit may determine the number of divisions of the convex hull region based on the length of the longer side of the circumscribing rectangle of the convex hull.

[0017] The recognition unit may recognize that the vessel is moving in a straight line if the area ratio of the area of ​​the contour region enclosed by the contour to the area of ​​the convex hull region enclosed by the convex hull is greater than the turning threshold, and may recognize that the vessel is turning if the area ratio is less than or equal to the turning threshold.

[0018] If the recognition unit determines that the vessel is not turning, it may recognize the direction in which the longer side of the circumscribing rectangle of the convex hull extends as the direction of travel of the vessel.

[0019] If the recognition unit determines that the vessel is turning, it may recognize the direction in which the side of the approximate triangle approximating the convex hull region that is closest to the centroid of the contour region extends as the direction of the vessel's movement.

[0020] A ship recognition method according to one aspect of the present invention involves acquiring an image captured from above, extracting the outline of a ship including its wake based on the image, determining the convex hull of the ship including its wake based on the image, and recognizing the ship based on the outline and the convex hull.

[0021] A ship recognition program according to one aspect of the present invention involves causing a computer to perform the following processes: acquire an image taken from above; extract the outline of a ship including its wake based on the image; determine the convex hull of the ship including its wake based on the image; and recognize the ship based on the outline and the convex hull.

[0022] According to the present invention, the movement of a ship can be recognized from captured images.

[0023] This figure shows a schematic configuration of a ship recognition system according to an embodiment. This is a functional block diagram of a ship recognition device according to the first embodiment. This is a flowchart for explaining the ship recognition method according to the first embodiment. This is a first example showing the contour and convex hull of a moving ship. This is a second example showing the contour and convex hull of a moving ship. This is a third example showing the contour and convex hull of a moving ship. This is a first example showing the contour and convex hull of a stationary ship. This is a second example showing the contour and convex hull of a stationary ship. This is a third example showing the contour and convex hull of a stationary ship. This figure shows the calculation result of the ratio of the contour area to the convex hull area for a moving ship. This figure shows the calculation result of the ratio of the contour area to the convex hull area for a stationary ship. This is a functional block diagram of a ship recognition device according to the second embodiment. This is a flowchart for explaining the ship recognition method according to the second embodiment. This is a diagram for explaining the ship recognition method according to the second embodiment. This is a functional block diagram of a ship recognition device according to the third embodiment. This is a flowchart for explaining the ship recognition method according to the third embodiment. This is a diagram for explaining the ship recognition method according to the third embodiment. This is a diagram for explaining the method for calculating the rotation angle of a straight-line vector according to the third embodiment. This is a flowchart illustrating the method for calculating the rotation angle of the turning vector according to the third embodiment. This is a diagram illustrating the method for calculating the rotation angle of the turning vector according to the third embodiment. This is a diagram illustrating the method for calculating the rotation angle of the turning vector according to the third embodiment. This is a diagram illustrating the various waves associated with a navigating vessel.

[0024] When a ship navigates on the water, a wake is generated. The shape of the wake, which appears as a physical phenomenon, can be described as geometric features and processed as image data. As shown in Figure 17, known types of wakes include the V-shaped bow Kelvin wave formed based on the interaction between the bow and the water surface, the V-shaped stern Kelvin wave formed based on the interaction between the stern and the water surface, the free-surface shock wave, which is a steep nonlinear wave generated around a ship moving at high speed, and the turbulent wake, which is formed from the stern by cavitation of the propeller and extends in a straight line.

[0025] Although there are subtle differences depending on the ship's shape, propulsion system characteristics, and water surface conditions, in simple terms, a wake wave can be approximated as a V-shaped figure extending from the hull (bow and stern) and a figure extending in a straight line from the stern. The cloudy area associated with wave breaking generally has a complex shape when examined in detail. Geometrically extracting the contour of this area reveals it to be detected as a concave polygon with numerous vertices. When the convex hull of the detected vertices is determined, the area of ​​the region enclosed by the convex hull (convex hull region) is larger than the area enclosed by the contour line (contour region) by the area of ​​the concave portion corresponding to the difference. Therefore, if the area of ​​the convex hull region is larger than the area of ​​the contour region, it can be inferred that a wake wave is being generated due to the navigation of a ship.

[0026] Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[0027] <Ship Recognition System 1> Referring to Figure 1, the schematic configuration of the ship recognition system 1 according to this embodiment will be described.

[0028] The ship recognition system 1 comprises a ship recognition device 10, a terminal 20 for user use, and an aerial imaging device 30. These components are connected via a communication network N for communication.

[0029] The ship recognition device 10 is configured to recognize ships based on images captured from above by an aircraft 30 or the like. For example, the ship recognition device 10 recognizes the movement of a ship (whether it is moving or stationary), the position of the ship (position of the bow), and / or the direction of the ship's course. In this application, "images captured from above" are not limited to images captured by a moving object such as an aircraft 30, but may also be images captured by equipment installed on top of a fixed structure such as a high-rise building. Details of the ship recognition device 10 will be described later.

[0030] Terminal 20 is configured to display images of the ship S captured by the aircraft 30, etc., and to display the recognition results from the ship recognition device 10. Terminal 20 can be, for example, a laptop computer, a desktop computer, or a tablet device.

[0031] The flying object 30 is equipped with a camera and is configured to photograph the water surface from above. In Figure 1, the flying object 30 is an artificial satellite, but it is not limited to this and may be an aircraft, helicopter, drone, balloon, etc.

[0032] <Ship Recognition Device According to the First Embodiment (Recognition of Ship Movement)> The ship recognition device 10 according to the first embodiment will be described in detail with reference to Figure 2.

[0033] The ship recognition device 10 comprises a communication unit 11, a storage unit 12, and a processing unit 13. These components are connected to each other by a bus 14.

[0034] The communication unit 11 is an interface for sending and receiving information with the terminal 20, aircraft 30, etc., via wireless or wired communication. The method and specifications of communication by the communication unit 11 are not particularly limited.

[0035] The storage unit 12 stores various types of information used by the processing unit 13, such as programs executed by the processing unit 13 and data used. Here, the data used includes images received from the aircraft 30, etc. The storage unit 12 is composed of a semiconductor memory and / or a hard disk drive, etc.

[0036] The processing unit 13 includes an image acquisition unit 131, a ship detection unit 132, a contour extraction unit 133, a convex hull creation unit 134, and a motion recognition unit 135. Each of these functional units is stored in the storage unit 12 in the form of a program that can be executed by a computer. For example, the processing unit 13 is composed of one or more processors, and by reading and executing programs from the storage unit 12, it realizes the functions corresponding to each program. One or more of the functions of the processing unit 13 may be realized by hardware.

[0037] Next, we will describe the various functional units of the processing unit 13.

[0038] The image acquisition unit 131 acquires images taken from above, specifically aerial images such as satellite images and aerial images taken by the aircraft 30, or images taken by equipment installed on top of high-rise buildings or towers. In this embodiment, the image acquisition unit 131 acquires images transmitted from the aircraft 30, etc., via the communication network N. The image acquisition unit 131 is not limited to acquiring images in real time from the aircraft 30, etc., but may also acquire images stored in an image server (not shown) connected to the communication network N.

[0039] The ship detection unit 132 detects ships in the images acquired by the image acquisition unit 131. The method for detecting ships is not particularly limited. For example, ship detection may be performed using artificial intelligence (AI). That is, ships in images may be detected using a pre-trained model (such as R-CNN (Region Based Convolutional Neural Network)) generated in advance by machine learning such as deep learning. When performing machine learning, the accuracy of detecting ships, including wakes, can be improved by including images of moving ships (i.e., ships and wakes) in the training data. Alternatively, ships may be detected using methods that detect multiple objects in an image in a single process, such as YOLO (You Only Look Once) or SSD (Single Shot Multibox Detector). In addition to AI-based methods, ships may be detected using techniques that analyze the characteristics of areas with locally different brightness distributions, or by using template matching techniques that calculate the similarity between a template image of a ship and a target image, and identify areas showing high similarity as ships. Furthermore, the accuracy of ship detection may be improved by applying preprocessing such as binarization, edge enhancement, LUT (Look Up Table), and filtering.

[0040] If the image acquired by the image acquisition unit 131 includes a ship, the contour extraction unit 133 extracts the outline of the ship, including its wake, based on the image. A wake is a white wave that appears behind a moving ship, and is also called a wake.

[0041] The contour extraction unit 133 extracts the contour of the ship, for example, by edge detection. The contour of the ship may be extracted using known library functions such as the findContours() function of OpenCV. Alternatively, functions such as FindImageShapes and EdgeDetect from the Wolfram language may be used.

[0042] The convex hull creation unit 134 obtains the convex hull of the ship including the wake based on the image acquired by the image acquisition unit 131. For example, based on the extracted contour, the convex hull of the ship including the wake is obtained. In this case, the convex hull is obtained by eliminating the concavities and convexities of the contour.

[0043] Note that the convex hull creation unit 134 may obtain the convex hull using a known library function such as the cv2.convexHull() function of OpenCV. Also, the convex hull may be obtained by an algorithm such as Graham Scan or Quick Hull.

[0044] The dynamic / static recognition unit 135 recognizes the dynamic / static state of the ship based on the contour and the convex hull. In the present embodiment, the dynamic / static recognition unit 135 recognizes the dynamic / static state of the ship based on the area of the region surrounded by the contour (contour area) and the area of the region surrounded by the convex hull (convex hull area). For example, the dynamic / static recognition unit 135 calculates the contour area and the convex hull area, and recognizes that the ship is moving when the value obtained by dividing the contour area by the convex hull area (contour area / convex hull area; hereinafter also referred to as "area ratio") is smaller than the threshold value, and recognizes that the ship is stopped otherwise. Note that the calculation of the contour area and the convex hull area can use a known library function such as the cv2.contourArea() function of OpenCV. The dynamic / static recognition unit 135 is an example of the recognition unit in the claims.

[0045] <First Embodiment of Ship Recognition Method> Referring to the flowchart of FIG. 3, an example of the ship recognition method (dynamic / static recognition) by the ship recognition device 10 according to the first embodiment will be described.

[0046] Step S1: The image acquisition unit 131 acquires an aerial image. In the present embodiment, the image acquisition unit 131 acquires an aerial image in real time from the aircraft 30. Note that the image acquisition unit 131 may acquire an image captured by a device installed on the top of a fixed structure such as a high-rise building.

[0047] Step S2: The ship detection unit 132 detects ships in the image acquired in step S1. If a ship is detected in the image (S2: Yes), the process proceeds to step S3. On the other hand, if no ship is detected in the image (S2: No), the process returns to step S1.

[0048] Step S3: The contour extraction unit 133 extracts the contour of the ship including the wake based on the image in which the ship was detected in step S2.

[0049] Step S4: The convex hull creation unit 134 obtains the convex hull of the ship including the wake. In the present embodiment, the convex hull creation unit 134 obtains the convex hull of the ship including the wake based on the contour extracted in step S3.

[0050] Step S5: The motion / static recognition unit 135 calculates the contour area (the area of the region surrounded by the contour) and the convex hull area (the area of the region surrounded by the convex hull). In the present embodiment, the motion / static recognition unit 135 calculates the value obtained by dividing the contour area by the convex hull area (area ratio).

[0051] Step S6: The motion / static recognition unit 135 determines whether the area ratio calculated in step S5 is smaller than the threshold value. If the area ratio is smaller than the threshold value (S6: Yes), the motion / static recognition unit 135 recognizes that the ship is moving. On the other hand, if the area ratio is greater than or equal to the threshold value (S6: No), the motion / static recognition unit 135 recognizes that the ship is stationary. The threshold value is, for example, a value of 0.6 or more and 0.8 or less.

[0052] In step S6, when the motion / static recognition unit 135 recognizes (determines) that the ship is moving, the ship recognition device 10 may notify the terminal 20 to that effect. Also, at this time, the ship recognition device 10 may transmit an image including the ship recognized as moving to the terminal 20.

[0053] The above processing flow is only an example, and various modifications are possible. When a plurality of ships are detected in the image in step S2, the processes of steps S3 to S8 are performed for each ship. Note that the area calculation in step S5 may be performed by a functional unit other than the motion / static recognition unit 135 (such as an area calculation unit not shown).

[0054] <Effects of the First Embodiment> As described above, according to the first embodiment, when an image includes a ship, the outline and convex hull of the ship, including its wake, are determined based on the image, and the movement of the ship is recognized based on the outline and convex hull. By using the outline and convex hull in this way, the movement of the ship can be accurately recognized. Furthermore, since the outline and convex hull of the ship can be extracted easily and quickly using library functions, the movement of the ship can be recognized in real time.

[0055] Figures 4A, 4B, and 4C show examples of aerial images of a moving vessel, respectively. Figures 5A, 5B, and 5C show examples of aerial images of a stationary vessel, respectively. In each figure, the symbol L1 indicates the contour line of the vessel S and the wake W, and the symbol L2 indicates the convex hull of the vessel S and the wake W. In step S3, the contour line L1 is determined, and in step S4, the convex hull L2 is determined. As can be seen from Figures 4A, 4B, 4C, 5A, 5B, and 5C, in a stationary vessel, there is almost no difference between the area of ​​the region enclosed by the contour line L1 and the area of ​​the region enclosed by the convex hull L2, whereas in a moving vessel, the difference between the area of ​​the region enclosed by the contour line L1 and the area of ​​the region enclosed by the convex hull L2 is large. The same can be said for the perimeter of the region enclosed by the contour line L1 and the convex hull L2.

[0056] Figure 6A shows the area ratios calculated based on the images in Figures 4A, 4B, and 4C. Figure 6B shows the area ratios calculated based on the images in Figures 5A, 5B, and 5C. As can be seen from these results, the area ratio for moving vessels is less than 0.5, which is smaller than the aforementioned threshold (0.6 to 0.8). On the other hand, the area ratio for stationary vessels is 0.9 or greater, which is larger than the threshold. By comparing the area ratio with the threshold in this way, the movement of vessels can be accurately grasped.

[0057] In the description of the first embodiment above, the movement of the vessel was recognized using the area ratio, but the value obtained by subtracting the contour area from the convex hull area (hereinafter referred to as "area difference") may also be used. In this case, the movement recognition unit 135 recognizes that the vessel is moving if the area difference is greater than a threshold, and recognizes that the vessel is stationary otherwise.

[0058] Alternatively, the movement of a vessel may be recognized using the length of the contour or convex hull (the perimeter of the region) rather than the area of ​​the region enclosed by the contour or convex hull. In this case, the movement recognition unit 135 recognizes that the vessel is moving if the difference between the length of the contour and the length of the convex hull is greater than a threshold, and recognizes that the vessel is stationary otherwise. The number of vertices constituting the contour or convex hull may be used as the length of the contour or convex hull.

[0059] Furthermore, the ship detection unit 132 may process the image based on meteorological and / or oceanographic information in the area corresponding to the image. Meteorological and / or oceanographic information may include, for example, information on typhoons, fronts, waves, and swells, and may be obtained from the websites of the Japan Meteorological Agency, private weather companies, local governments, etc. The image processing may be performed before the ship detection process, or it may be performed on the image in which a ship has been detected. By processing the image in this way, noise components such as waves are removed from the image acquired by the image acquisition unit 131, and the contour area and convex hull area can be calculated accurately, thereby improving the accuracy of ship movement.

[0060] Furthermore, the motion recognition unit 135 may estimate the speed of the vessel based on the contour and the convex hull. For example, the motion recognition unit 135 estimates that the vessel is moving at a faster speed if the area ratio between the region enclosed by the contour and the region enclosed by the convex hull is small (as it approaches 0), the area difference is large, and / or the difference in length (circumference) is large.

[0061] <Ship Recognition Device According to the Second Embodiment (Forward and Rear Recognition of Ships)> The ship recognition device according to the second embodiment will be described in detail with reference to Figure 7. The ship recognition device 10 according to the second embodiment is configured to recognize the forward and rear of the ship (bow and stern) in addition to the movement of the ship. The second embodiment will be described below, focusing on the differences from the first embodiment.

[0062] The ship recognition device 10 according to the second embodiment includes a communication unit 11, a storage unit 12, and a processing unit 13A. These components are connected to each other by a bus 14.

[0063] The processing unit 13A includes an image acquisition unit 131, a ship detection unit 132, a contour extraction unit 133, a convex hull creation unit 134, a motion / station recognition unit 135, and a front / back recognition unit 136. Each of these functional units is stored in the storage unit 12 in the form of a program that can be executed by a computer. The functional units other than the front / back recognition unit 136 are the same as in the first embodiment, so a detailed explanation is omitted.

[0064] The front / rear recognition unit 136 divides the convex hull region into a plurality of divided regions along its short axis, and recognizes the front and rear of the vessel based on the first and second divided regions that are closest to the ends in the long axis direction. Note that the front / rear recognition unit 136 is an example of a recognition unit in the claims.

[0065] In this embodiment, the front / rear recognition unit 136 calculates the area ratio of the contour region to the area of ​​the convex hull region for the first and second divided regions, respectively, and recognizes the front and rear of the vessel based on the area ratio of the first and second divided regions. When the area ratio is obtained by dividing the area of ​​the contour region by the area of ​​the convex hull region, the front / rear recognition unit 136 recognizes the first divided region as the front region and the second divided region as the rear region if the area ratio of the first divided region is greater than the area ratio of the second divided region. Conversely, when the area ratio of the first divided region is less than the area ratio of the second divided region, the unit recognizes the first divided region as the rear region and the second divided region as the front region.

[0066] Here, the contour region of the first divided region is the region enclosed by the contour line and dividing line (dividing line DL described later) of the first divided region, and the contour region of the second divided region is the region enclosed by the contour line and dividing line of the second divided region. Furthermore, the convex hull region of the first divided region is the region enclosed by the convex hull line and dividing line of the first divided region, and the convex hull region of the second divided region is the region enclosed by the convex hull line and dividing line of the second divided region.

[0067] In this way, the front and rear recognition unit 136 recognizes the front and rear of the vessel based on the area ratio of the contour region area to the area of ​​the convex hull region, calculated for the first and second divided regions located at both ends in the direction of the major axis of the convex hull region, which are among the multiple regions obtained by dividing the convex hull region surrounded by the convex hull line L2 along the direction of the minor axis perpendicular to the major axis (principal axis) of the convex hull region.

[0068] The front / back recognition unit 136 may determine the number of divisions of the convex hull region based on the length of the longer side of the circumscribing rectangle of the convex hull. This ensures sufficient convex recognition accuracy even when the trailing wave is long. The circumscribing rectangle can be obtained using known library functions such as the OpenCV's cv2.boundingRect() function and cv2.minAreaRect().

[0069] Furthermore, the front / rear recognition unit 136 may determine the number of divisions of the convex hull region based on the ratio of the short side to the long side of the circumscribing rectangle. For example, if the long side is considerably longer than the short side, the number of divisions may be increased. Alternatively, the front / rear recognition unit 136 may divide the convex hull region into predetermined length intervals along the long side of the circumscribing rectangle.

[0070] Furthermore, the division of the convex hull region is not limited to using a circumscribed rectangle. For example, the major axis of the convex hull region can be found using a function such as cv2.fitLine(), and its perpendicular bisector can be used as the dividing line DL. In this case as well, there may be two or more divisions. That is, the dividing line DL may be a line perpendicular to the point that trisects (or passes through) the major axis segment of the convex hull region (or quarters, etc.).

[0071] <Ship Recognition Method According to the Second Embodiment> Referring to the flowchart in Figure 8, an example of a ship recognition method (front and rear recognition) using the ship recognition device 10 according to the second embodiment will be described. The method described below is performed, for example, after step S7 described in the first embodiment.

[0072] In the following description of the ship recognition method according to the second embodiment, it is assumed that in step S1 described in the first embodiment, the image acquisition unit 131 acquires an aerial image of the ship shown in Figure 9A. In Figure 9A, reference numeral L1 indicates the contour lines of the ship S and wake W, and reference numeral L2 indicates the convex hull of the ship S and wake W. In step S3 described in the first embodiment, the contour line L1 is determined, and in step S4, the convex hull L2 is determined.

[0073] As can be seen from Figure 9A, in the forward direction of the ship S, the area where the waves break is small, so there is little difference between the contour area and the convex hull area (i.e., the area ratio is small). On the other hand, in the backward direction of the ship, the area where the waves break is large, so there is a large difference between the contour area and the convex hull area, and the area ratio is large.

[0074] Step S21: The front / rear recognition unit 136 divides the region enclosed by the convex hull (convex hull region). In this embodiment, as shown in Figure 9B, the convex hull region is divided into two regions, division region A1 and division region A2. In Figure 9B, the division line DL indicates the division line that divides the convex hull region. In this step, for example, a rectangle circumscribed around the convex hull line L2 (circumscribed rectangle) is found, and the convex hull region is divided by a line parallel to the shorter side of that rectangle that bisects the convex hull region. Alternatively, the convex hull region may be divided by the perpendicular bisector of the side connecting the midpoints of the shorter sides of the circumscribed rectangle.

[0075] Furthermore, if the trailing wave is long, dividing the convex hull region into two parts may result in insufficient difference in area ratio between the divided regions, potentially leading to inadequate recognition accuracy. For this reason, the number of divisions in the convex hull region may be determined based on its length. For example, if the length of the convex hull region is greater than or equal to a first length, it may be divided into three parts, and if the length of the convex hull region is greater than or equal to a second length (longer than the first length), it may be divided into four parts.

[0076] Alternatively, the number of divisions in the convex hull region may be determined based on the length of the longer side of the circumscribing rectangle. This length may also be measured using the number of pixels in the aerial image.

[0077] Step S22: The motion recognition unit 135 calculates the contour area and convex hull area for each divided region (divided regions A1 and A2).

[0078] Step S23: The motion recognition unit 135 calculates the area ratio of each divided region (divided regions A1 and A2). In this embodiment, the area ratio is the same as in the first embodiment, obtained by dividing the contour area by the convex hull area (i.e., contour area / convex hull area).

[0079] Step S24: The motion recognition unit 135 determines whether the area ratio of divided region A1 and the area ratio of divided region A2 calculated in step S22 are different. If the area ratio of divided region A1 and the area ratio of divided region A2 are different (S24: Yes), the unit proceeds to step S25. On the other hand, if the area ratios of each divided region are the same (S24: No), the unit determines that the front and rear of the vessel cannot be recognized (step S26).

[0080] Step S25: The front / rear recognition unit 136 recognizes the division region with the larger area ratio as the front region and the division region with the smaller area ratio as the rear region. In the example image in Figure 9A, the area ratio of division region A1 is 0.66 and the area ratio of division region A2 is 0.29, so it is determined that division region A1 is the front region and division region A2 is the rear region.

[0081] In addition, in the above method, the processing in steps S22 to S24 may be performed by the front / rear recognition unit 136.

[0082] <Effects of the Second Embodiment> As described above, according to the second embodiment, for an aerial image including a ship S, the convex hull region is divided into divided region A1 and divided region A2, the area ratio of each divided region A1 and A2 is calculated, and the divided region with the larger area ratio is recognized as the forward region, and the region with the smaller area ratio is recognized as the rear region. This makes it possible to recognize the front and rear of a moving ship S. For example, the bow of the ship S can be easily identified. Furthermore, since the division of the convex hull region can be easily and quickly performed using library functions, the front and rear of the ship can be recognized in real time.

[0083] In the description of the second embodiment above, the front and rear of the vessel were recognized using the area ratio, but the area difference obtained by subtracting the contour area from the convex hull area may also be used. In this case, the front and rear recognition unit 136 recognizes divided region A1 as the front region and divided region A2 as the rear region when the area difference of divided region A1 is smaller than the area difference of divided region A2.

[0084] Alternatively, the front-to-back recognition of a vessel may be performed using the length of the contour or convex hull (the perimeter of the region) rather than the area of ​​the region enclosed by the contour or convex hull. In this case, the front-to-back recognition unit 136 recognizes divided region A1 as the front region and divided region A2 as the rear region if the difference between the length of the contour and the length of the convex hull of divided region A1 is smaller than the difference between the length of the contour and the length of the convex hull of divided region A2.

[0085] <Ship Recognition Device According to the Third Embodiment> The ship recognition device according to the third embodiment will be described in detail with reference to Figure 10. The ship recognition device 10 according to the third embodiment is configured to recognize the direction of travel of a ship in addition to recognizing the movement and front / rear of the ship. The third embodiment will be described below, focusing on the differences from the first and second embodiments.

[0086] The ship recognition device 10 according to the third embodiment includes a communication unit 11, a storage unit 12, and a processing unit 13B. These components are connected to each other by a bus 14.

[0087] The processing unit 13B includes an image acquisition unit 131, a ship detection unit 132, a contour extraction unit 133, a convex hull creation unit 134, a motion / station recognition unit 135, a front / rear recognition unit 136, and a direction of travel recognition unit 137. Each of these functional units is stored in the storage unit 12 in the form of a program that can be executed by a computer. The functional units other than the direction of travel recognition unit 137 are the same as in the second embodiment, so a detailed explanation is omitted.

[0088] The direction of travel recognition unit 137 recognizes whether a vessel is turning or not based on the area ratio of the area of ​​the contour region enclosed by the contour to the area of ​​the convex hull region enclosed by the convex hull. In this embodiment, the direction of travel recognition unit 137 determines that the vessel is not turning if the area ratio obtained by dividing the area of ​​the contour region by the area of ​​the convex hull region is greater than the turning threshold, and determines that the vessel is turning if the area ratio is less than or equal to the turning threshold. Note that the direction of travel recognition unit 137 is an example of a recognition unit in the claims.

[0089] If it is determined that the vessel is not turning, the direction of travel recognition unit 137 recognizes the direction of the straight-line vector Vs (described later) as the direction of travel. The direction of the straight-line vector Vs is the major axis direction of the convex hull (for example, the direction in which the long side of the circumscribing rectangle of the convex hull region extends). On the other hand, if it is determined that the vessel is turning, the direction of travel recognition unit 137 recognizes the direction of the turning vector Vt (described later) as the direction of travel. The speed of the vessel S may also be estimated based on the magnitudes of the straight-line vector Vs and the turning vector Vt.

[0090] <Ship Recognition Method According to the Third Embodiment> Referring to the flowchart in Figure 11, an example of a ship recognition method (direction of travel recognition) using the ship recognition device 10 according to the third embodiment will be described. The method described below is performed, for example, after step S7 described in the first embodiment, or after step S25 described in the second embodiment.

[0091] In the following description of the ship recognition method according to the third embodiment, it is assumed that in step S1, the image acquisition unit 131 acquires an aerial image of the ship shown in Figure 12. However, in step S33, it is assumed that an aerial image of the ship shown in Figure 15 has been acquired. In Figures 12 and 15, the reference numeral L1 indicates the contour lines of the ship S and the wake W, and the reference numeral L2 indicates the convex hull of the ship S and the wake W. In step S3, as described in the first embodiment, the contour line L1 is determined, and in step S4, the convex hull L2 is determined.

[0092] Step S31: The direction of travel recognition unit 137 determines whether the area ratio calculated in step S5 of the first embodiment is greater than the turning threshold. If the area ratio is greater than the turning threshold (S31: Yes), the direction of travel recognition unit 137 determines that the vessel S is not turning and calculates the rotation angle of the straight-ahead vector of the vessel S (Step S32). On the other hand, if the area ratio is less than or equal to the turning threshold (S31: No), the direction of travel recognition unit 137 determines that the vessel S is turning and calculates the rotation angle of the turning vector (Step S33). Details of Steps S32 and S33 will be described below. Note that the turning threshold may be smaller than the threshold used in Step S6 of the first embodiment.

[0093] Step S32: The direction of travel recognition unit 137 calculates the rotation angle of the straight-line vector Vs of the ship S. In this embodiment, as shown in Figure 13, a circumscribed rectangle R is found that circumscribes the convex hull region enclosed by the convex hull L2, and the direction in which the longer side of the circumscribed rectangle R extends is taken as the direction of the straight-line vector Vs. The rotation angle of the straight-line vector Vs is then considered as the rotation angle of the ship S. That is, the angle between the straight-line vector Vs and the coordinate axis (θ or θ + 180°) is taken as the rotation angle of the ship S.

[0094] Step S33: The direction of travel recognition unit 137 calculates the rotation angle of the turning vector Vt of the ship S. An example of how to calculate the rotation angle of the turning vector Vt will be explained with reference to the flowchart in Figure 14. It is assumed that in step S1, the image acquisition unit 131 acquired the aerial image of the ship shown in Figure 15.

[0095] Step S41: The direction of travel recognition unit 137 calculates the centroid G of the region enclosed by the contour line L1 (contour region), as shown in Figure 16. The centroid G may be calculated using a known library function such as the cv2.moments() function of OpenCV. In addition, the centroid G may be calculated for the forward region described in the second embodiment.

[0096] Step S42: The direction of travel recognition unit 137 calculates a triangle that approximates the convex hull region (convex hull approximation triangle) as shown in Figure 16. Contour approximation may be performed using known library functions such as the cv2.approxPolyDP() function of OpenCV, and the approximation triangle may be obtained. In this embodiment, a convex hull approximation triangle T that approximates the convex hull region is calculated.

[0097] Step S43: The direction of travel recognition unit 137 identifies the side of the convex hull approximation triangle T calculated in step S42 that is the shortest distance from the centroid G of the contour region, and calculates the angle θ of that side as the rotation angle of the turning vector Vt. The direction in which the side of the convex hull approximation triangle T that is closest to the centroid G of the contour region extends is the direction of the turning vector Vt.

[0098] Step S34: The direction of travel of the vessel S is determined based on the rotation angle calculated in step S32 or step S33 and the direction in which the aerial image was taken.

[0099] When step S32 is executed, the direction of travel recognition unit 137 recognizes that the rotation angle θ + 180° is the direction of travel of the vessel S when the area recognized as the front area by the front / rear recognition unit 136 is on the left side of the circumscribed rectangle R, as shown in Figure 13. On the other hand, when the area recognized as the front area by the front / rear recognition unit 136 is on the right side of the circumscribed rectangle R, the direction of travel recognition unit 137 recognizes that the rotation angle θ is the direction of travel of the vessel S.

[0100] In Figure 13, in accordance with actual ship operation practices, the upper part of the aerial image (smaller y-coordinate) is set to 0° (north), the right side (larger x-coordinate) to 90° (east), the lower part (larger y-coordinate) to 180° (south), and the left side (smaller x-coordinate) to 270° (west). The same applies to Figure 16, which will be discussed later.

[0101] When step S33 is executed, the direction of travel recognition unit 137 recognizes the rotation angle θ° of the turning vector Vt as the direction of travel of the vessel S when the area recognized as the forward area by the front / rear recognition unit 136 is on the right side of the convex hull approximation triangle T. On the other hand, when the area recognized as the forward area by the front / rear recognition unit 136 is on the left side of the convex hull approximation triangle T, the direction of travel recognition unit 137 recognizes the rotation angle θ + 180° as the direction of travel of the vessel S.

[0102] In the example shown in Figure 16, the bearing of the ship S calculated from the rotation angle of the circumscribed rectangle R (i.e., the rotation angle of the straight-line vector Vs) is 78.4°, whereas the bearing of the ship S calculated using the method described above (i.e., the rotation angle of the turning vector) is 99.7°, which is a value closer to the actual value.

[0103] <Effects of the Third Embodiment> As described above, according to the third embodiment, the direction of travel recognition unit 137 recognizes that the vessel S is moving in a straight line (not turning) if the area ratio of the area of ​​the contour region enclosed by the contour line L1 to the area of ​​the convex hull region enclosed by the convex hull L2 is greater than the turning threshold for an aerial image including the vessel S, and recognizes that the vessel S is turning if the area ratio is less than or equal to the turning threshold. When the direction of travel recognition unit 137 determines that the vessel S is moving in a straight line, it recognizes the direction of travel of the vessel S using the straight-line vector Vs, and when it determines that the vessel S is turning, it recognizes the direction of travel of the vessel S using the turning vector Vt. This makes it possible to recognize the direction of travel of a moving vessel S. Furthermore, since the processing of this embodiment can be easily and quickly executed using library functions, the direction of travel of a vessel can be recognized in real time.

[0104] Note that the calculation of the turning vector is not limited to the method described in steps S41 to S43. For example, a triangle approximating the forward region of the convex hull described in the second embodiment may be found, and the vertex of that triangle furthest from the dividing line DL may be considered as the position of the bow, and the direction from the center of gravity G to the bow may be calculated as the turning vector. In this case, the center of gravity G may be the center of gravity of the forward region.

[0105] The first to third embodiments have been described above. For example, if you want to know whether or not a ship is moving, you only need to implement the first embodiment. If you want to know the general direction of travel in addition to the ship's movement, you should implement the first and second embodiments in order. If you want to know the detailed direction of travel in addition to the ship's movement, you should implement the first, second, and third embodiments in order. The first to third embodiments may be implemented in any combination, or each embodiment may be implemented independently.

[0106] Based on the above description, those skilled in the art may conceive of additional effects and various modifications of the present invention, but the embodiments of the present invention are not limited to the individual embodiments described above. Components from different embodiments may be combined as appropriate. Various additions, modifications, and partial deletions are possible without departing from the conceptual idea and spirit of the present invention derived from the claims and their equivalents.

[0107] Furthermore, each part of the processing unit 13 may be implemented by a processor within the ship recognition device 10 executing a predetermined program, with the software processing being concretely realized using hardware resources, or it may be implemented by the hardware itself.

[0108] Furthermore, the configuration shown in Figure 2 is just one example, and at least some of the components of the ship recognition device 10 may be provided within the terminal 20.

[0109] Furthermore, at least a part of the ship recognition device 10 described in the above-described embodiment may be configured as hardware or as software. In the case of software configuration, a program that realizes at least a part of the functions of the ship recognition device 10 may be stored on a recording medium such as a flexible disk or CD-ROM, and loaded into a computer for execution. The recording medium is not limited to removable ones such as magnetic disks or optical disks, but may also be a fixed recording medium such as a hard disk drive or memory.

[0110] Furthermore, a program that implements at least some of the functions of the ship recognition device 10 may be distributed via communication lines such as the Internet (including wireless communication). In addition, the program may be encrypted, modulated, or compressed and distributed via wired or wireless lines such as the Internet, or stored on a recording medium.

[0111] 1. Ship Recognition System 10. Ship Recognition Device 11. Communication Unit 12. Memory Unit 13, 13A, 13B. Processing Unit 14. Bus 131. Image Acquisition Unit 132. Ship Detection Unit 133. Contour Extraction Unit 134. Convex Hull Creation Unit 135. Motion Recognition Unit 136. Front / Rear Recognition Unit 137. Direction of Travel Recognition Unit 20. Terminal 30. Aircraft A1, A2. Divided Region DL. Dividing Line G. Center of Gravity L1. Contour Line L2. Convex Hull N. Communication Network R. Circumscribed Rectangle S. Ship T. Convex Hull Approximation Triangle Vs. Straight Direction Vector Vt. Turning Vector W. Wake

Claims

1. A ship recognition device comprising: an image acquisition unit that acquires an image taken from above; a contour extraction unit that extracts the outline of a ship, including its wake, based on the image; a convex hull creation unit that determines the convex hull of the ship, including its wake, based on the image; and a recognition unit that recognizes the ship based on the outline and the convex hull.

2. The apparatus according to claim 1, wherein the recognition unit recognizes the movement of the vessel based on the first area of ​​the contour region enclosed by the contour and the second area of ​​the convex hull region enclosed by the convex hull.

3. The apparatus according to claim 2, wherein the recognition unit recognizes that the vessel is moving if the value obtained by dividing the first area by the second area is less than a threshold, and recognizes that the vessel is stopped otherwise.

4. The apparatus according to claim 2, wherein the recognition unit recognizes that the vessel is moving if the value obtained by subtracting the first area from the second area is greater than a threshold, and recognizes that the vessel is stopped otherwise.

5. The apparatus according to claim 1, comprising a ship detection unit for detecting ships in the acquired image, wherein when the ship detection unit detects a ship in the image, the contour extraction unit extracts the contour and the convex hull creation unit determines the convex hull.

6. The apparatus according to claim 5, wherein the ship detection unit processes the image based on information regarding weather and / or sea conditions in the region corresponding to the image.

7. The apparatus according to claim 1, wherein the recognition unit estimates the speed of the vessel based on the contour and the convex hull.

8. The apparatus according to claim 1, wherein the recognition unit divides the convex hull region surrounded by the convex hull into a plurality of divided regions along its short axis, and recognizes the front and rear of the vessel based on the first and second divided regions among the plurality of divided regions that are closest to the ends of the convex hull region in the long axis direction.

9. The apparatus according to claim 8, wherein the recognition unit calculates an area ratio for the first divided region and the second divided region by dividing the area of ​​the contour region by the area of ​​the convex hull region, and if the area ratio of the first divided region is greater than the area ratio of the second divided region, it recognizes the first divided region as the front region and the second divided region as the rear region.

10. The apparatus according to claim 8, wherein the recognition unit determines the number of divisions of the convex hull region based on the length of the longer side of the circumscribing rectangle of the convex hull.

11. The apparatus according to claim 1, wherein the recognition unit recognizes that the vessel is moving in a straight line if the area ratio of the area of ​​the contour region enclosed by the contour to the area of ​​the convex hull region enclosed by the convex hull is greater than the turning threshold, and recognizes that the vessel is turning if the area ratio is less than or equal to the turning threshold.

12. The apparatus according to claim 11, wherein, if the recognition unit determines that the vessel is not turning, it recognizes the direction in which the longer side of the circumscribing rectangle of the convex hull extends as the direction of travel of the vessel.

13. The apparatus according to claim 11, wherein, when the recognition unit determines that the vessel is turning, it recognizes the direction in which the side of the approximate triangle approximating the convex hull region that is closest to the centroid of the contour region extends as the direction of travel of the vessel.

14. A ship recognition method comprising: acquiring an image taken from above; extracting the outline of a ship including its wake based on the image; determining the convex hull of the ship including its wake based on the image; and recognizing the ship based on the outline and the convex hull.

15. A ship recognition program that causes a computer to perform the following processes: acquire an image taken from above; extract the outline of a ship, including its wake, based on the image; determine the convex hull of the ship, including its wake, based on the image; and recognize the ship based on the outline and the convex hull.

Images