Method for determining a ship-water interface, and method and system for determining a positional relationship between a self-ship and a target ship

Abstract

A method for determining a ship-water interface (40) between at least one target ship (34) and a water surface (22) of a water area in which the target ship (34) sails from an image (20) showing the target ship (34) on the water area, the method comprising: determining, for pixels of the image (20), an interface value representing a probability of a corresponding pixel indicating the ship-water interface (40); determining, from the image data, boundary box data of a boundary box (32) surrounding the target ship (34) in the image (20); and determining ship-water interface data representing a position and an extent of the ship-water interface (40) in the image (20) according to the determined interface value and the boundary box data.

Description

Technical Field

[0001] The present invention relates to the field of ship-water interface detection and marine range and bearing measurement. In particular, the present invention relates to a method for determining a ship-water interface, as well as a method and system for determining the positional relationship between a self-ship and a target ship.

Background Art

[0002] Autonomous solutions are expected to transform the world of international transportation. One or more cameras as sensors for collecting real-world information can play an important role in the development of these solutions, for example, any ship, i.e., a vessel or a boat, can be operated successfully by relying on visual perception. Regarding vision, a monocular system is more attractive than a stereo system because it is less complex and less expensive, for example, compared to a stereo system. However, regarding ship positioning, including, for example, the range to a self-ship, i.e., the owned ship, and / or the bearing of a target ship, i.e., another ship, a monocular system can be limited to the positioning of points along the horizon, i.e., points on the water surface of the water area where the ship is sailing. Since the extraction of these points can be difficult, some methods known in the art assume that the ship-water interface is aligned with the lower edge of the bounding box generated by an object detection algorithm configured to detect ships in the image. Another common assumption in the art can be that the midpoint of the lower end of the bounding box is on the ship-water interface. However, these assumptions can introduce inaccuracies, for example, there can be applications where the determined ship-water interface has to be determined more accurately if it is to be used to determine the positional relationship between a self-ship and a target ship, for example, the range between the self-ship and the target ship, in other words the distance, and / or the bearing of the target ship.

[0003] Therefore, there is a need for a more accurate method for determining the ship-water interface within an image and / or for determining the positional relationship between the own ship and the target ship. SUMMARY OF THE INVENTION

[0004] An object of the present invention is to provide a method for determining the ship-water interface between at least one target ship and the water surface of a water area, which can be carried out in a very accurate and / or easy, simple, and / or cost-effective manner, and / or which can make it possible to very accurately determine the positional relationship between the target ship and the own ship.

[0005] An object of the present invention is to provide a method for determining the positional relationship between an own ship and a target ship within a water area, which can be carried out in a very accurate and / or easy, simple, and / or cost-effective manner.

[0006] An object of the present invention is to provide a system for determining the positional relationship between an own ship and a target ship within a water area, which can be operated in a very accurate and / or easy, simple, and / or cost-effective manner.

[0007] These objects are achieved by the subject matter of the independent claims. Further exemplary embodiments are apparent from the dependent claims and the following description.

[0008] The object is achieved by a method for determining the ship-water interface between at least one target ship and the water surface of the water area in which the target ship sails from an image showing the target ship on the water area, the method comprising receiving image data of the image, determining an interface value for a pixel of the image, the interface value representing the probability that the corresponding pixel indicates the ship-water interface, determining boundary box data of a boundary box surrounding the target ship in the image from the image data, and determining ship-water interface data, the ship-water interface data representing the position and extent of the ship-water interface in the image according to the determined interface value and boundary box data.

[0009] The boundary box data can be determined before, after, or simultaneously with the interface value. If there are two or more target vessels in the image, more boundary boxes may be determined accordingly, for example, one for each target vessel. In this context, determining the boundary boxes enables, in particular, separating two different target vessels from each other or distinguishing a buoy from a vessel, especially when these vessels overlap each other in the image. Preferably, the boundary box data is determined such that the target vessel fits exactly into the corresponding boundary box, as is normal for boundary boxes determined by object detection algorithms or instance segmentation as known in the art.

[0010] Determining the interface value and determining the ship - water interface data depending on the interface value and the boundary box data enables a very accurate determination of the ship - water interface and can be performed in an easy, simple, and / or cost - efficient manner. As a result, the above - described method enables a very accurate determination of the positional relationship between the target vessel and the own vessel, as will be explained below.

[0011] The water area may be a lake, ocean, sea, or river. The image data may be received by an entity that executes the above - described method. The entity may be, for example, a general - purpose computer on the own vessel or a system for determining the positional relationship between the own vessel and the target vessel in the image.

[0012] Preferably, at least one interface value is determined for each of the pixels of the image. The interface value may be a probability value ranging from 0 to 1 or from 0% to 100%. Alternatively, the interface value may be convertible from or to these probability values, for example, it may range from 0 to 1, 0 to 10, -5 to 5, or from negative to positive infinity. The interface value may be presented in the form of one interface value per pixel. For example, if the interface value ranges from 0 to 1, the threshold may be set to 0.5, and all pixels having an interface value greater than 0.5 represent the corresponding pixels indicating the ship-water interface, and all pixels having an interface value of 0.5 or less represent the corresponding pixels not indicating the ship-water interface. It should be noted that the above ranges and thresholds are only examples, and other ranges and / or thresholds may be set to distinguish pixels indicating the ship-water interface from pixels not indicating the ship-water interface.

[0013] The interface value may be determined by semantic segmentation of the image and optionally by post-processing the output of the semantic segmentation. The semantic segmentation may be performed by a neural network. The neural network may be trained and / or configured such that the neural network outputs a ship-water value representing the probability of the corresponding pixel of an image indicating a target ship, water, or neither, for example, void, air, land, etc. These ship-water values may be modified by one or more post-processing steps such that the modified output of the neural network represents the probability of the corresponding pixel indicating whether or not there is a ship-water interface. In this embodiment, the modified ship-water value corresponds to the interface value. This neural network may sometimes be referred to hereinafter as the first neural network. The first neural network may be trained by a training data set including image data of a certain amount of images showing ships on a water area, and the corresponding ships and water surfaces in the water area are labeled in the image.

[0014] Alternatively, the neural network may be trained and / or configured such that the output of the neural network directly represents the probability of the corresponding pixel, regardless of whether the output of the neural network represents the ship-water interface. In this embodiment, the output of the neural network directly corresponds to the interface value. Thus, in this embodiment, no post-processing of the output of the neural network is required to obtain the interface value. This neural network may be referred to as a second neural network. The second neural network may be trained by a training data set including image data of a quantity of images showing ships on water, and the ship-water interface is labeled within the images. As an alternative to using the quantity of images to train the neural network, the neural network may be trained by one-shot learning, as is known in the art.

[0015] According to an embodiment, the interface value is interpreted as a cost or a reward, and the ship-water interface data is determined by minimizing the cost or maximizing the reward, respectively. The cost may be minimized or the reward may be maximized by a seam finder module, and the interface value may be used as an input for the seam finder module. The ship-water interface data may include x-y coordinates or pairs of the number of columns and rows of pixels of an image showing the ship-water interface within the image. The seam finder module may use, for example, dynamic programming or, as is known in the art, one of other methods from graph theory for determining ship-water interface data, such as Dijkstra's algorithm.

[0016] According to an embodiment, the interface values form an array of interface values, and the positions of the interface values within the array correspond to the positions of the corresponding pixels in the image. The cost and reward each refer to the sum of the interface values of all the pixels required to proceed pixel by pixel from one side of the array corresponding to one side of the bounding box to another side of the array corresponding to another side of the bounding box. Figuratively speaking, the seam finder module can be executed from one side of the bounding box to the other side in order to minimize the cost or maximize the reward. One side can correspond to the left side of the bounding box, and the other side can correspond to the right side of the bounding box. In other words, when calculating the cost, the seam finder module may search for the least expensive path from the left side of the bounding box to the right side of the bounding box, or when calculating the reward, the seam finder module may search for the most valuable path from the left side of the bounding box to the right side of the bounding box, and the result is interpreted as the ship-water interface. Alternatively, the seam finder module may be executed from the right side of the bounding box to the left side.

[0017] According to an embodiment, the bounding box data is determined by an object detection algorithm configured to detect ships in the image and determine ship-water interface data for each determined bounding box. The object detection algorithm may be any object detection algorithm known in the art that can detect ships in the image. In particular, the object detection algorithm may be another neural network trained to detect ships in the image by a training data set including a certain amount of images showing ships, especially ships on the corresponding waters. The object detection algorithm for determining the bounding box data may sometimes be referred to as the third neural network hereinafter.

[0018] Alternatively, according to another embodiment, the bounding box data is determined by instance segmentation. In this case, the bounding box data may be extracted from the output of the instance segmentation as described later.

[0019] As an alternative to the first or second neural network using semantic segmentation to determine the ship-water value or interface value, respectively, at a certain step and using instance segmentation to determine the third neural network or bounding box data at another step, the first and / or second neural network may use panoptic segmentation to determine the ship-water value or interface value and the bounding box data in one step and / or by the corresponding neural network only, without particularly requiring the third neural network or instance segmentation.

[0020] According to an embodiment, after receiving the image data and before determining the ship-water interface data, the method further includes determining a ship-water value for pixels of the image from the image data, where the ship-water value represents the probability that the corresponding pixel is a target ship, water area, or neither, i.e., indicates voids, air, land, etc., and determining an interface value from the ship-water value. In particular, the ship-water value may be corrected by one or more post-processing steps, where the input of this post-processing is the ship-water value and the output of this post-processing is the interface value. Alternatively, the ship-water value may be used as an input to another neural network, hereinafter referred to as the fourth neural network, which determines the interface value from the ship-water value. The ship-water value may be determined by semantic segmentation or panoptic segmentation of the image.

[0021] Semantic segmentation or panoptic segmentation that outputs ship-water values may be performed by a first neural network. The ship-water values may be presented in the form of a triplet of three ship-water values per pixel. The first ship-water value having the first index within the triplet may represent the probability that the corresponding pixel shows neither water nor a ship. The second ship-water value having the second index within the triplet may represent the probability that the corresponding pixel shows the water surface. The third ship-water value having the third index within the triplet may represent the probability that the corresponding pixel shows the target ship. Preferably, at least one ship-water value is determined for each pixel of the image. The ship-water values may be probability values ranging from 0 to 1 or from 0% to 100%. Alternatively, the ship-water values may be convertible from or to these probability values, and may reach, for example, from 0 to 1, from 0 to 10, from -5 to 5, or from negative to positive infinity.

[0022] The first neural network can be pre-trained using a training data set of image data of a certain amount of images showing ships on water areas, by supervised learning, and by images labeled with respect to the corresponding ships and water areas. If a fourth neural network determines an interface value from the ship-water values output by the first neural network, the fourth neural network can be pre-trained by supervised learning using, for example, a training data set containing a large array of ship-water values determined by the first neural network from the image data, and by an array labeled with respect to the ship-water values corresponding to those pixels of the image showing the ship-water interface.

[0023] Alternatively, when the second neural network determines the interface value directly from the image data, the second neural network may be pre-trained by supervised learning using a training data set of image data of a certain amount of images showing ships on the water area, with the images labeled with respect to the corresponding ship-water interface. In particular, the second neural network may be trained such that only the physical ship-water interface can be determined as the ship-water interface by the correspondingly labeled images. Thus, virtual ship-water interfaces shown in the images but not corresponding to the real-world ship-water interface may not be determined as the ship-water interface by the second neural network trained accordingly. Instead of using supervised learning with a large amount of training data, one or more machine learning techniques that do not require a large amount of training data, particularly image data or ship-water values, such as one-shot learning as known in the art, may be used to train one or more of the above neural networks.

[0024] According to an embodiment, the method further includes determining an edge value of a pixel of the image after receiving the image data and before determining the ship-water interface data, where the edge value represents a pixel corresponding to whether or not an edge in the image is shown, and determining the ship-water interface data according to the edge value. This may contribute to improving the accuracy of the ship-water interface data. The edge value may be determined by any image processing algorithm configured for edge detection, as known in the art.

[0025] According to an embodiment, the ship-water interface data is determined according to the edge value by multiplying the interface value with the corresponding edge value on a pixel-by-pixel basis and determining the ship-water interface data according to the corresponding product.

[0026] According to an embodiment, the method further includes determining that all pixels whose corresponding interface value is below a predetermined threshold do not represent the ship-water interface. Thus, before outputting the position and extent of the ship-water interface in the form of ship-water interface data, the ship-water interface data determined by the seam finder module may be modified such that all pixels whose corresponding interface value is below the predetermined threshold are determined not to represent the ship-water interface.

[0027] According to an embodiment, the interface value and / or the ship-water value is determined by a neural network, such as a second neural network, or a first neural network and / or a fourth neural network, as described above.

[0028] According to an embodiment, the method further comprises modifying the image data according to the determined ship-water interface data such that the ship-water interface is depicted within the image when the image is presented on a display. For example, the image data may be modified according to the ship-water interface data such that the ship-water interface is highlighted within the image after modification when the image is presented on a display. The ship-water interface can be easily displayed within the image. However, displaying the ship-water interface within the image may not be necessary for all applications that use the determined ship-water interface. For example, when the ship-water interface data is used to determine the positional relationship between the own ship and a target ship on the water area, the ship-water interface need not be displayed on the display.

[0029] The object is achieved by a method for determining the positional relationship between a own ship and a target ship within a body of water. The own ship is equipped with a camera for taking an image of the surroundings of the own ship, and the method comprises determining at least one ship-water interface within the image according to the method described above, determining position data representing the position and the orientation of the camera when the camera captured the image, determining pixel data of at least one image point on the ship-water interface within the image from the image data, determining the real-world coordinates of at least one real-world point of the ship-water interface on the body of water, and determining the positional relationship between the own ship and the target ship according to the position data and the real-world coordinates of at least one real-world point of the ship-water interface on the water surface.

[0030] It should be understood that the features of a method for determining a ship-water interface between at least one target ship and the water surface of a body of water as described above and below can be features of a method for determining the positional relationship between an own ship and a target ship within the body of water as described above and below.

[0031] At least one ship-water interface within the image can be determined by determining the corresponding ship-water interface data as described above. The camera position may correspond to the position of the center of the camera. The camera center may be given by the pinhole of the camera. The camera can be calibrated before starting one of the methods described above. In particular, the orientation and / or position of the camera relative to the own ship can be calibrated by any calibration method known to those skilled in the art. In particular, the intrinsic and extrinsic calibration of the camera may be performed as described, for example, in the section "Camera Models and Calibration" of "Learning OpenCV" (published by O'Reilly Media, Inc., ISBN: 9780596516130, released in September 2008) by Gary Bradski and Adrian Kaehler. When the camera is calibrated, since there is a fixed spatial relationship between the camera and the own ship, the position data also represents the position of the own ship.

[0032] When performing the above method, a self-ship motion correction may be performed to take into account the movement of the self-ship when determining the ship-water interface and / or the positional relationship between the self-ship and the target ship. The self-ship motion correction may be performed by any self-ship motion correction method known in the art.

[0033] The real-world coordinates of the real-world points of the ship-water interface on the water area can be determined by any projection method known in the art, for example, as described in detail in textbooks on computer vision, such as "Multiple View Geometry in Computer Vision" by Richard Hartley and Andrew Zisserman, 2nd Edition, isbn-13 978-0-521-54051-3, for example, Part I: Camera Geometry and Single View Geometry.

[0034] According to an embodiment, the real-world coordinates of the real-world points are determined by extrapolating the camera position of the camera through the image points to the water surface of the water area according to the position data and the pixel data. To extrapolate the camera position to the water surface through the image points of the image, the image may be virtually placed in an image plane, for example, a virtual image plane, or the real plane of the camera, or any other suitable plane. The image plane of the camera may correspond to the sensor plane of the camera, for example, the CCD plane of the camera if the camera is a CCD camera. The virtual image plane may be arranged between the camera and the water area. In contrast, the image plane may be arranged behind the camera when viewed from the water area, in which case the real-world points may be determined by extrapolating the camera position through the image points to the water surface.

[0035] According to an embodiment, the positional relationship refers to the range, in other words the distance, between the own ship and the target ship, and / or the bearing of the target ship. The bearing of the target ship may be a relative bearing or an absolute bearing. The relative bearing refers to the angle between the heading of the own ship and the line extending from a predetermined point on the own ship to a determined real-world point on the ship-water interface. The absolute bearing refers to the angle between the basic direction of north (hereinafter referred to as "true north") and the line extending from a predetermined point on the own ship, for example the camera position or the center of the own ship, to a determined real-world point on the ship-water interface. In this context, the absolute bearing may be regarded as the positional relationship between the own ship and the target ship. This is because usually the heading of the own ship relative to true north is known, and thus when the absolute bearing of the target ship is known, the relative bearing between the own ship and the target ship is also known.

[0036] According to an embodiment, the pixel data of at least one image point is determined such that the image point corresponds to the real-world point on the ship-water interface that is closest to the own ship. The real-world point on the ship-water interface that is closest to the own ship can be determined by projecting all the image points of the ship-water interface in the image onto a real-world coordinate system. Then, the point in the real-world coordinate system that is closest to the own ship can be determined. When the real-world point on the ship-water interface that is closest to the own ship is determined in the real-world coordinate system, the corresponding image point in the image coordinate system can be determined because there is a point-to-point correspondence between the image point in the image coordinate system and the corresponding real-world point in the real world.

[0037] The object is achieved by a system for determining the positional relationship between an own ship and a target ship in a body of water, the system comprising a camera for capturing an image of the surroundings of the own ship, the camera being arranged, for example installed, on the own ship, and a processing unit coupled to the camera and configured to execute the above-described method for determining the positional relationship between the own ship and the target ship in the body of water. Optionally, the processing unit may be arranged remotely from the camera and / or the ship. For example, the processing unit may be arranged in a computer of the ship or in a server remote from the ship.

[0038] Features of a method for determining a ship-water interface between at least one target ship and the water surface of a water area, and / or features of a method for determining a positional relationship between a self-ship and a target ship in a water area as described above and below, may be features of a system for determining a positional relationship between a self-ship and a target ship in a water area as described above and below, and it must be understood.

[0039] Furthermore, a computer program may be provided, and the computer program includes instructions configured to execute at least one of the above methods when executed by a processor of a computer or by a processing unit of the above system. It should be understood that features of the method as described above and below may be features of a control device and / or a power supply system as described above and below.

[0040] Also, a computer-readable medium storing the above computer program may be provided. It should be understood that features of the method as described above and below may be features of a computer-readable medium as described above and below. The computer-readable medium may be a floppy (registered trademark) disk, a hard disk, a USB (Universal Serial Bus) storage device, a RAM (Random Access Memory), a ROM (Read Only Memory), an EPROM (Erasable Programmable Read Only Memory), or a flash memory. The computer-readable medium may be a data communication network, such as the Internet, that enables the download of program code. The computer-readable medium may be a non-temporary or temporary computer-readable medium.

[0041] These and other aspects of the present invention will become apparent from the embodiments described below and will be clarified with reference to those embodiments.

Brief Description of the Drawings

[0042] The subject matter of the present invention will be described in more detail below with reference to exemplary embodiments illustrated in the accompanying drawings.

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

[0043] The reference signs used in the drawings and their meanings are listed in summary form in the list of reference signs. In principle, in the drawings, the same reference signs are assigned to the same parts.

Embodiments for Carrying Out the Invention

[0044] FIG. 1 shows a block diagram illustrating an exemplary embodiment of a method for determining a ship-water interface 40 (see FIG. 4) between at least one target ship 34 (see FIG. 2) and the water surface 22 of the water area within an image 20 (see FIG. 2). This method can be executed by a processing unit of a general-purpose computer or, as will be described later, by a processing unit of a system for determining the positional relationship between a self-ship (not shown) and a target ship 34 in a water area. Hereinafter, the target ship 34 will be referred to simply as a "ship", while the ship on which a camera (not shown) for capturing the image 20 is arranged will be referred to as a "self-ship". The water area may be a lake, ocean, sea, or river.

[0045] In a first step of this method, the image 20 may be captured by a camera (not shown).

[0046] FIG. 2 shows an example of an image 20 showing a ship 34 in a water area. In particular, it can be seen from the image 20 that the ship 34 is navigating on the water area. The water area may be represented in the image 20 by the water surface 22 of the water area. The image 20 may show the water surface 22, land 24, for example, two pieces of land, air 26, and the ship 34. Only one continuous water surface 22 is shown in the image 20. Another valid image 20 may show two or more water surfaces 22 separated from each other. Only one ship 34 is shown in the image 20. Another valid image 20 may show two or more ships 34 separated from each other or overlapping each other.

[0047] The camera may be arranged on the own ship. The camera may generate image data representing the image 20. The image data may be transmitted to, and / or received by, a processing unit that executes a method for determining the ship-water interface 40. The image data may be stored in the memory of a general-purpose computer or system, the memory being coupled to the processing unit. When the method is executed, the processing unit may receive the image data directly from the camera or from the memory. The camera may be calibrated with respect to the position and / or orientation of the camera relative to the own ship. In particular, the camera may be calibrated by external and internal calibration. The position of the camera may include the height of the camera on the own ship. Thus, if the position and orientation of the own ship are known, the position and orientation of the camera are also known.

[0048] The first neural network B2 can be configured to receive the image data and determine at least one ship-water value for each pixel of the image 20 from the image data. For example, the first neural network B2 may be configured to determine three or more ship-water values for each pixel of the image. In particular, when the image 20 includes an array of N×M pixels, where N and M are natural numbers, the first neural network B2 can determine a 3×N×M ship-water value assigned to the corresponding pixel. The ship-water value represents the probability that the corresponding pixel represents a target ship, the water surface 22, or neither, for example, land 24, air 26, or a void. The ship-water value can be determined by semantic segmentation of the image 20. The semantic segmentation may be performed, for example, by a first neural network B2 trained for distinguishing water and ships in the image by supervised learning using a correspondingly labeled image. For example, the first neural network B2 may be pre-trained by supervised training using a certain amount of images 20 showing ships 34 on the water area and by images 20 labeled with respect to the corresponding water surface 22 and ships 34. In addition, images 20 that do not show any ships 34 may also be involved in the training of the first neural network B2. Instead of using an amount of images 20 for training, one-shot learning as known in the art may be applied to train the first neural network B2.

[0049] Figure 3 shows an example of the visualization 36 of the ship-water value determined from the image 20 of FIG. 2 by the first neural network B2. In particular, FIG. 3 shows the representation of the first segmentation output of the first neural network B2. The first segmentation output corresponds to the ship-water value. Regarding the output of the first neural network B2, the final layer of the first neural network B2 may be, for example, a softmax function. The softmax function may provide an output for each pixel of the image 20 that can correspond to a probability triplet, for example (x0, x1, x2), where the first value of the triplet with index 0 may represent the probability that the corresponding pixel shows neither the water surface 22 nor the ship 34, the second value of the triplet with index 1 may represent the probability that the corresponding pixel shows a part of the water surface 22, and the value of the triplet with index 2 may represent the probability that the corresponding pixel shows a part of the target ship 34. The ship-water value may be a probability value ranging from 0 to 1 or from 0% to 100%. Alternatively, the ship-water value may be convertible from or to these probability values, for example, it may range from 0 to 1, 0 to 10, -5 to 5, or from negative to positive infinity. When these ship-water values are between 0 and 1, all the ship-water values of one triplet sum to 1. Thus, for example, if the output of one of the pixels is (0.1, 0.7, 0.2), the probabilities that the pixel is neither part of the water surface 22 nor part of the ship 34, and that it is neither part of the water surface 22 nor part of the ship 34 are 0.1, 0.7, and 0.2, respectively. If more classes are required than just the water surface, the ship, or neither of them, for example for the sky 26 or the land 24, more than three ship-water values may be determined per pixel accordingly.

[0050] For example, as shown in FIG. 3, in order to visualize the output of semantic segmentation, the output of the first neural network B2, i.e., the ship-water value, may be assigned different colors. For example, the index of the ship-water value within the triplet may be assigned to the corresponding pixel representing the ship-water value with the highest probability within the corresponding triplet. For example, if the first ship-water value of the triplet has the highest probability within the triplet, "0" may be assigned to the corresponding pixel; if the second ship-water value of the triplet has the highest probability within the triplet, "1" may be assigned to the corresponding pixel; if the third ship-water value of the triplet has the highest probability within the triplet, "2" may be assigned to the corresponding pixel. In the above example, the ship-water value with index 1, i.e., 0.7, is the highest value of the triplet, and this ship-water value represents the probability that the corresponding pixel indicates a part of the water surface 22. Therefore, in the above example, the value "1" may be assigned to the corresponding pixel indicating a part of the water surface 22. And the numbers "0", "1", "2" assigned to the pixels may be assigned different colors respectively. For example, if the number assigned to one of the pixels is "1", the first color, for example, green, may be used for the corresponding pixel for visualization 36; if the number assigned to that pixel is "2", the second color, for example, red, may be used for the corresponding pixel for visualization 36.

[0051] As described above, the argmax function is applied to assign the index of the ship-water value triplet to the corresponding pixel representing the ship-water value of the corresponding triplet with the highest probability. In particular, the ship-water value may be input to the argmax function. Next, the argmax function outputs the index of the ship-water value representing the highest probability within the corresponding triplet for each corresponding pixel, and colors may be assigned to that output to achieve visualization 36.

[0052] The post - processing module B4 may receive the output of the first neural network B2, i.e., an array of ship - water values, and may be configured to determine at least one interface value for each ship - water value, and thus for each pixel of the image 20. Thus, the interface value can be determined by the post - processing module B4. In particular, continuing with the above example, if the image 20 contains an array of N×M pixels and the output of the first neural network B2 provides a corresponding array of ship - water values of 3×N×M, i.e., a 3D array, the post - processing module B4 determines an array of N×M interface values, i.e., a 2D array, assigned to the corresponding pixels and / or ship - water values. The interface value represents the probability that the corresponding pixel represents the ship - water interface 40 (see FIG. 4) between the ship 34 and the water surface 22.

[0053] Thus, the ship - water values can form a 3D array. The first two dimensions of this 3D array may correspond to the spatial dimensions of the image 20, and the last dimension may refer to three channels corresponding to the ship 34 and the water surface 22, or may refer to neither. The post - processing module B4 can determine the interface value from the ship - water values by extracting the ship - water values corresponding to the ship 34 and corresponding to the water surface 22 from the 3D array. The extracted values form two 2D arrays, one for the ship - water values corresponding to the ship 34 and one for the ship - water values corresponding to the water surface 22, and the two dimensions of the 2D arrays correspond to the spatial dimensions of the image 20. These 2D arrays can then be multiplied pixel - by - pixel with each other to obtain another 2D array, and the resulting 2D array contains one interface value per pixel, and the multiplication is defined as the Hadamard product. The two dimensions of the resulting 2D array may correspond to the spatial dimensions of the image 20.

[0054] For example, assume that one of the pixels has a ship - water value of water = 0.4, ship = 0.5, and "neither" = 0.1. Then, for this pixel, the ship - water value representing the probability that the pixel represents the water surface 22 and the ship - water value representing the probability that the pixel represents the ship 34 are multiplied by each other, i.e., 0.4 * 0.5 = 0.2. Thus, in the resulting 2D array, the interface value of the pixel under consideration is 0.2. This procedure can be repeated for all pixels within the image 22 or at least within the area of interest, for example, within the bounding box 32.

[0055] However, since there are many ways to define a reasonable function for converting three ship - water values per pixel into a single interface value per pixel, many alternative solutions are possible. For example, an alternative solution can be based on modeling the ship - water values using a Dirichlet distribution.

[0056] The interface value may be a probability value ranging from 0 to 1 or from 0% to 100%. Alternatively, the interface value may be convertible from or to these probability values, for example, it may range from 0 to 10, - 5 to 5, or from negative to positive infinity. Instead of determining the interface value from the ship - water value by the post - processing described above, the interface value may be determined by another neural network (not shown), for example, a fourth neural network. The fourth neural network may be trained by a training data set containing ship - water values of an array of a certain amount of ship - water values. Instead of using the amount of the array, one - shot learning as known in the art may be applied when training other neural networks.

[0057] Figure 4 shows an example of visualization 38 of interface values determined from the ship - water values visualized in Figure 3. In particular, Figure 4 shows the output of post - processing module B4. The output corresponds to the interface values. The interface values may be binary values, where binary 1 may be assigned to all pixels indicating the ship - water interface, and binary 0 may be assigned to all pixels not indicating the ship - water interface. Then, to provide visualization 38, a first color may be assigned to all pixels indicating the ship - water interface 40, a second color may be assigned to all pixels not indicating the ship - water interface 40, or no color may be assigned.

[0058] The ship - water interface 40 may comprise a physical ship - water interface 42 and a virtual ship - water interface 44. The physical ship - water interface 42 corresponds to the ship - water interface in the real world where the water surface 22 physically contacts the ship 34 directly. The virtual ship - water interface 44 corresponds to the ship - water interface 40 in the image 20 where the pixels indicating the water surface 22 are the next adjacent pixels to the pixels indicating the ship 34, and in the real world, the corresponding area of the surface of the ship 34 has no direct contact with the water surface 22.

[0059] Figure 5 shows an example of an image 20 showing the target ship 34 in the water area according to Figure 2 and the bounding box 32 around the target ship 34.

[0060] An object detection module B10 may be provided and may be configured to determine bounding box data of a bounding box 32 that surrounds a ship 34 in an image 20 from image data. The bounding box data may be determined by an object detection algorithm represented by the object detection module B10. The bounding box data may be determined before, after, or simultaneously with the interface value. If there are two or more target ships 34 in the image 20, more bounding boxes 32 may be determined accordingly, for example, one bounding box 32 for each target ship 34. The object detection algorithm may be any conventional object detection algorithm capable of detecting a ship 34 in the image 20. The object detection algorithm may be, for example, a neural network called a third neural network. The third neural network may be pre-trained so as to be able to detect and mark a ship 34 in the image 20. For example, the third neural network may be trained by a training data set including a corresponding amount of labeled images or by one-shot learning as known in the art. The bounding box data may be matched with the image data such that the bounding box 32 surrounds the ship 34 in the image 20 when the image 20 is displayed on a display. Preferably, the bounding box data is determined such that the corresponding ship 34 exactly fits into the corresponding bounding box 32. This is normal for a bounding box determined by an object detection algorithm as known in the art. Alternatively or additionally, the bounding box data may be matched with the ship-water interface data such that the bounding box 32 surrounds the ship-water interface 40 in the visualization 39 of the ship-water interface 40 when the visualization 39 of the ship-water interface 40 is displayed on a display.

[0061] As an alternative to using an object detection algorithm to determine the bounding box data, the bounding box data can be determined by instance segmentation. In this case, the bounding box data can be extracted from the output of the instance segmentation. For example, the outermost pixels in the x and y directions of the target ship identified by the instance segmentation may be used to determine the bounding box data, thereby determining the bounding box around the target ship. In particular, the bounding box data can be determined such that the corresponding bounding box includes these outermost pixels. The instance segmentation algorithm that performs the instance segmentation may be any instance segmentation algorithm known in the art that can detect ships in an image. In particular, the instance segmentation algorithm may be another neural network trained to detect ships in an image by a training data set that includes a certain amount of images showing ships, particularly ships on the corresponding waters. A suitable example of instance segmentation is described in the paper "Mask R-CNN" (2017) by He, K., Gkioxari, G., Dollar, P. and Girshick, R. in the Proceedings of the IEEE International Conference on Computer Vision (pp. 2961-2969).

[0062] Instead of the first or second neural network using semantic segmentation to determine the ship-water value or interface value, respectively, at one step and instance segmentation to determine the third neural network or bounding box data at another step, the first and / or second neural network may use panoptic segmentation, which enables the ship-water value or interface value and the bounding box data to be determined, respectively, in one step and / or by only the corresponding neural network, without the need for a third neural network or instance segmentation in particular. Basically, panoptic segmentation is a combination of instance segmentation and semantic segmentation. Thus, panoptic segmentation can be used to replace both object detection and semantic segmentation. For example, panoptic segmentation may be used as described in the paper "UPSNet: A Unified Panoptic Segmentation Network" by Yuwen Xiong et al., CVPR 2019.

[0063] The seam finder module B6 can be configured to determine ship-water interface data representing the ship-water interface 40 according to the interface values provided by the post-processing module B4 and the bounding box data provided by, for example, the object detection module B10. To determine the ship-water interface data from the interface values, the interface values are interpreted as costs or rewards, and the ship-water interface data is determined by minimizing the cost or maximizing the reward, respectively. The cost may be minimized or the reward may be maximized by the seam finder module B6. In this embodiment, the interface values may be used as inputs for the seam finder module B6. As described above, the interface values form a 2D array of interface values, and the positions of the interface values within the array correspond to the positions of the corresponding pixels within the image 20. The ship-water interface data determined by the seam finder module B6 may comprise the x-y coordinates of the pixels of the image 20 indicating the ship-water interface 40 within the image 20, or pairs of column and row numbers. The ship-water interface data may represent the position and extent of the ship-water interface 40 within the image 20. The seam finder module B6 can use, for example, seam curving, dynamic programming, or one of other methods known in the art from graph theory, such as Dijkstra's algorithm, to determine the ship-water interface data.

[0064] Costs and rewards refer to the sum of the interface values of all pixels necessary to move in pixel units from one side of the array corresponding to one side of the boundary box 32 to the other side of the array corresponding to the other side of the boundary box 32. In other words, the seam finder module B6 can perform from one column of the array of interface values to the other column in order to minimize costs or maximize rewards. One column corresponds to the left side of the boundary box 32, and the other column corresponds to the right side of the boundary box 32. Figuratively speaking, when calculating costs, the seam finder module B6 may search for the least expensive path from the left side of the boundary box 32 to the right side of the boundary box 32, and when calculating rewards, may search for the most valuable path from one column of the array to the other column of the array. Alternatively, the seam finder module B6 may be executed from the right side to the left side of the boundary box 32. When there are two or more vessels 34 and correspondingly two or more boundary boxes 32, the seam finder module B6 can be executed separately for each determined boundary box 32, that is, for each determined vessel 34, and for each boundary box 32, that is, for each vessel 34, the ship-water interface data can be determined.

[0065] FIG. 6 shows an example of a visualization 39 of a ship-water interface 40 determined from the interface values visualized in FIG. 4 and the boundary box data visualized as the boundary box 32 in FIG. 5, particularly by the seam finder module B6. In other words, FIG. 6 may show a visualization 39 of the output of the seam finder module B6. As can be seen from FIG. 6, at least a part of the virtual ship-water interface 44 may be removed from the ship-water interface 40 by the seam finder module B6 so that the determined ship-water interface 40 fits more accurately to the real world.

[0066] The output module B8 may be configured to output the ship-water interface 40, for example, in the form of a visualization 39 of the ship-water interface 40 within the image 20, or as raw data without particularly showing the visualization 39 of the ship-water interface 40 and / or the image 20 on a display.

[0067] Optionally, when the image 20 is shown on a display (not shown), the image data may be configured to be modified according to the determined ship-water interface data such that the ship-water interface 40 is shown within the image 20. For example, when the image 20 is shown on the display 10, the image data may be modified according to the ship-water interface data such that the ship-water interface is highlighted within the image after modification.

[0068] Accordingly, the ship-water interface 40 can be easily shown within the image 20 and / or within the visualization 39 of the ship-water interface 40. However, showing the ship-water interface 40 within the image 20 or within the visualization 39 of the ship-water interface 40 may not be necessary for all applications that use the determined ship-water interface 40. For example, when the ship-water interface data is used to determine the positional relationship between the own ship on the water area and the target ship 34, the ship-water interface 40 does not need to be shown on the display. In this case, the knowledge of the ship-water interface data may be sufficient to determine the positional relationship between the own ship and the target ship 34.

[0069] Optionally, an edge detection module B12 may be provided and may be configured to determine edge values from the image data. The edge detection module B12 can comprise any software for finding edges in the image 20, as is known in the art. For example, one possible edge detection method is described in "Digital Image Processing" by Rafael Gonzalez (author), Richard Woods (author), 3rd edition, ISBN 9780131687288, where the edge values may be obtained by calculating the magnitude of the gradient, as described in Section 10.2.5 "Basic Edge Detection", and may be combined with a threshold, as described in the same section. Alternatively, more advanced techniques, such as those described in Section 10.2.6 "More Advanced Techniques for Edge Detection", such as "The Canny edge detector", may be used. The output from the edge detection module B12 can be given by binary values, for example, 1 for all pixels indicating an edge and 0 for all pixels not indicating an edge. Alternatively, the magnitude of the edge can be used to represent the result of the edge detection. The magnitude of the edge is a measure of the edge strength of the corresponding pixel and is typically the magnitude of the gradient, providing a non-binary value in pixel units of the image 20. The resulting edge values can be multiplied by the interface values, for example, by pixel unit multiplication.

[0070] FIG. 7 shows a flowchart of an exemplary embodiment of a method for determining the positional relationship between the own ship in the water area and the target ship 34. The positional relationship between the own ship and the target ship 34 can refer to the range 50 (see FIG. 8) between the own ship and the target ship 34, and / or the azimuth RB, AB of the target ship 34 (see FIG. 8). To execute this method, the own ship can be equipped with a system for determining the positional relationship between the own ship and the ship 34 in the water area. The system includes a camera for capturing an image 20 around the own ship, and the camera is disposed on the own ship. The system can further include a processing unit coupled to the camera and configured to execute the following method. Before starting the method, the camera, particularly the orientation and / or position of the camera relative to the own ship, can be calibrated so that there is a fixed and / or known spatial relationship between the camera and the own ship. When executing this method, an own ship motion correction may be performed to take into account the movement of the own ship when determining the positional relationship with respect to the target ship 34.

[0071] In step S2, at least one ship-water interface 40 in the image 20 is determined by, for example, the above-described method described with respect to FIGS. 1 to 6, or by the following method described with respect to FIGS. 9 and 10. At least one ship-water interface 40 in the image 20 may be determined by determining corresponding ship-water interface data.

[0072] In step S4, position data is determined, and the position data can represent the camera position 28 (see FIG. 7) of the camera and the camera orientation of the camera when the camera captures the image 20. The camera position 28 of the camera may correspond to the position of the center of the camera. The camera center may be provided by the pinhole of the camera. The position data can also represent the position of the own ship, i.e., the own ship position 30, due to the fixed and / or known spatial relationship between the camera and the own ship.

[0073] FIG. 8 illustrates the principle of an exemplary embodiment of a method for determining the positional relationship between a own ship within a body of water and a target ship 34. From FIG. 8, it can be seen that the camera is arranged at the camera position 28. The camera can be calibrated such that the spatial relationship between the camera position 28 and the own ship position 30 of the own ship is fixed and / or known. For example, the projection of the camera position 28 onto the water surface 22 can be regarded as the own ship position 30. Alternatively, any other point on the water surface 22 having a fixed and / or known spatial relationship to the camera position 28, for example, the center of the own ship, can be regarded as the own ship position 30.

[0074] The image plane of the camera can correspond to the sensor plane of the camera, for example, the CCD plane of the camera if the camera is a CCD camera. The virtual image plane 52 may be arranged between the camera and the body of water. The visualization 39 or image 20 showing the ship-water interface 40 may be arranged on the virtual image plane 52 of the camera.

[0075] In step S6, the pixel data of at least one image point 46 on the ship-water interface 40 within the visualization 39 of the ship-water interface 40 or within the image 20 can be determined from the ship-water interface data or from the corrected image data. The pixel data of the at least one image point 46 may be determined such that the image point 46 corresponds to the real-world point 48 of the ship-water interface 40 that is closest to the own ship. The real-world point 48 of the ship-water interface 40 that is closest to the own ship can be determined by projecting all the image points of the ship-water interface 40 within the image 20 into the real-world coordinate system. Then, the point in the world coordinate system that is closest to the own ship can be determined. When the real-world point 48 of the ship-water interface 40 that is closest to the own ship is determined in the world coordinate system, a point-to-point correspondence exists between the image point and the corresponding real-world point, so that the corresponding image point in the image coordinate system can be determined.

[0076] In step S8, the real-world coordinates of at least one real-world point 48 of the ship-water interface 40 on the water area may be determined by extrapolating the camera position 28 of the camera through the image point 46 to the water surface 22 of the water area according to the position data and pixel data, and the image is within the virtual image plane 52 of the camera. The real-world coordinates of the real-world point 48 of the ship-water interface 40 on the water area can be determined by any projection method known in the art, for example, as described in detail in textbooks on computer vision, such as "Multiple View Geometry in Computer Vision" by Richard Hartley and Andrew Zisserman, 2nd edition, isbn-13 978-0-521-54051-3, for example, Part I: Camera Geometry and Single View Geometry.

[0077] In step S10, the positional relationship between the own ship and the target ship 34, that is, the range 50 and / or the azimuth RB, AB between the own ship and the target ship 34, can be determined according to the real-world coordinates of at least one real-world point 48 of the ship-water interface 40 on the water surface 22 and at the own ship position 30.

[0078] The azimuth of the target ship 34 may be the relative azimuth RB or the absolute azimuth AB. The relative azimuth RB may refer to or correspond to the angle between the bow azimuth 56 of the own ship, that is, the direction of the own ship with respect to true north 58, and a line 54 extending from a predetermined point on the own ship, for example, the own ship position 30 or the camera position 28 to the determined real-world point 48 of the ship-water interface 40. The absolute azimuth AB may refer to the angle between true north 58 and the line 54. In this context, the absolute azimuth AB can also be regarded as the positional relationship between the own ship and the target ship 34. This is because usually the bow azimuth 56 of the own ship with respect to true north 58 is known, and therefore, when the absolute azimuth AB of the target ship 34 is known, the relative azimuth RB between the own ship and the target ship 34 is also known.

[0079] FIG. 9 shows a block diagram illustrating an exemplary embodiment of a method for determining a ship-water interface 40 between at least one ship 34 and a body of water. Some of the method steps, and some of the modules that execute the corresponding method steps, may correspond to the method steps and modules described with respect to FIGS. 1-6. Accordingly, only the method steps and modules that are different between the methods described with respect to FIGS. 9 and 10 and the methods described with respect to FIGS. 1-6 are described below.

[0080] In this embodiment, a neural network, i.e., a second neural network B14, may receive the image data of the image 20 and may be configured to generate an interface value directly depending on the image data. In this context, the second neural network B14 may be configured to determine at least one interface value for each pixel of the image 20, the interface value representing the probability that the corresponding pixel indicates the ship-water interface 40. For example, the second neural network B14 may be pre-trained using a certain amount of images 20 showing a ship 34 on the body of water, and by images 20 labeled with respect to the corresponding ship-water interface 40, by supervised learning. In particular, the second neural network B14 may be trained to detect only the physical ship-water interface 42 and not to detect the virtual ship-water interface 44. Instead of using the amount of images 20 labeled for training the second neural network B14, the second neural network B14 may be trained by one-shot learning, as is known in the art.

[0081] FIG. 10 shows an example of a diagram showing the ship-water interface 40 determined by the second neural network B14, the virtual ship-water interface 44 being shown by a dashed line in FIG. 10 for information purposes only. The generated ship-water interface data may not include information corresponding to the virtual ship-water interface 44.

[0082] Optionally, a threshold module B16 may be arranged. The threshold module B16 may be configured to determine that all pixels whose corresponding interface value is below a predetermined threshold do not represent a ship-water interface. In other words, the threshold module B16 may be configured to set all interface values below a predetermined threshold to an interface value representing the probability that the corresponding pixel does not represent the ship-water interface 40, for example, zero. Therefore, before outputting the position and extent of the ship-water interface 40 in the form of ship-water interface data, the ship-water interface data determined by the second neural network B14 may be corrected by the threshold module B16 such that all pixels whose corresponding interface value is below a predetermined threshold are determined not to represent the ship-water interface 40.

[0083] The neural networks and modules described above, for example, the first neural network B2, the second neural network B14, the third and / or fourth neural networks, the seam finder module B6, the output module B8, the object detection module B10, the edge detection module B12, and / or the threshold module B16 may each be implemented by software, hardware, or a combination of software and hardware.

[0084] Although the present invention has been illustrated and described in detail in the drawings and the foregoing description, such illustrations and descriptions should be considered as illustrative or exemplary and not restrictive, and the present invention is not limited to the disclosed embodiments. Other variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed invention, from a consideration of the drawings, the disclosure, and the appended claims. In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality. A single processor or other unit may perform the functions of several items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be advantageously used. Any reference signs in the claims should not be construed as limiting the scope.

[0085] List of reference signs 20 Image 22 Water surface 24 Land 26 Air 28 Camera position 30 Own ship position 32 Bounding box 34 Target ship 36 Visualization of ship - water value 38 Visualization of interface value 39 Visualization of ship - water interface 40 Ship - water interface 42 Physical ship - water interface 44 Virtual ship - water interface 46 Image point 48 Real - world point 50 Range 52 Virtual image plane 54 Line 56 Leading own ship 58 True north AB Absolute bearing RB Relative bearing B2 First neural network B4 Post - processing module B6 Viewfinder Module B8 Output Module B10 Object Detection Module B12 Edge Detection Module B14 Second Neural Network B16 Threshold Module

Claims

1. A method for determining a ship-water interface (40) between at least one target ship (34) and the water surface (22) of a water area in which the target ship (34) sails, from an image (20) showing the target ship (34) on the water area, comprising: receiving image data of the image (20); determining an interface value for a pixel of the image (20), the interface value representing a probability that the corresponding pixel represents the ship-water interface (40); determining boundary box data of a boundary box (32) surrounding the target ship (34) in the image (20) from the image data; determining ship-water interface data according to the determined interface value and the boundary box data, the ship-water interface data representing the position and extent of the ship-water interface (40) in the image (20); A method comprising the above steps.

2. The method according to claim 1, wherein the interface value is interpreted as a cost or a reward, and the ship-water interface data is determined by minimizing the cost or maximizing the reward, respectively.

3. The interface values form an array of interface values, and the positions of the interface values in the array correspond to the positions of the corresponding pixels in the image. The cost and the reward refer to the sum or product of the interface values of all pixels required to move pixel by pixel from one side of the array corresponding to one side of the boundary box (32) to the other side of the array corresponding to the other side of the boundary box (32). The method according to claim 2.

4. The method according to any one of claims 1 to 3, wherein the boundary box data is determined by an object detection algorithm configured to detect ships in the image (20) or by instance segmentation.

5. After receiving the image data and before determining the ship-water interface data, determining a ship-water value of the pixel of the image (20) from the image data, the ship-water value representing a probability that the corresponding pixel represents the target ship (34), the water surface (22), or neither; determining the interface value from the ship-water value; The method according to any one of claims 1 to 4, further comprising the above steps.

6. After receiving the image data and before determining the ship-water interface data, Determining an edge value of the pixel of the image (20), the edge value indicating whether the corresponding pixel represents an edge in the image (20), and Determining the ship-water interface data according to the edge value; The method according to any one of claims 1 to 5, further comprising.

7. The ship-water interface data is determined according to the edge value, by multiplying the interface value by the corresponding edge value in pixel units and determining the ship-water interface data according to the corresponding product, or by adding the interface value to the corresponding edge value in pixel units and determining the ship-water interface data according to the corresponding sum, The method according to claim 6, which is determined.

8. Determining that all pixels whose corresponding interface value is below a predetermined threshold do not represent the ship-water interface (40); The method according to any one of claims 1 to 7, further comprising.

9. The interface value and / or the ship-water value is determined by a neural network, The method according to any one of claims 1 to 8.

10. The ship-water interface data includes the positions and / or coordinates of the pixels representing the ship-water interface (40) in the image (20), The method according to any one of claims 1 to 9.

11. When the image (20) is shown on a display, modifying the image data according to the determined ship-water interface data so that the ship-water interface (40) is illustrated in the image (20); The method according to any one of claims 1 to 10, further comprising.

12. A method for determining the positional relationship between a self-ship in the water area and a target ship (34) in the water area, the self-ship being equipped with a camera for capturing an image (20) around the self-ship, the method comprising: Determining at least one ship-water interface (40) in the image (20) according to any one of claims 1 to 10; Determining position data representing the camera position (28) and orientation of the camera when the camera captures the image (20); Determining pixel data of at least one image point (46) on the ship-water interface (40) in the image (20) from the image data; determining the real-world coordinates of at least one real-world point (48) of the ship-water interface (40) on the water area; determining the positional relationship between the own ship and the target ship (34) according to the position data and the real-world coordinates of the at least one real-world point (48) of the ship-water interface (40) on the water surface (22); A method comprising the steps.

13. The real-world coordinates of the real-world point (48) are determined by extrapolating the camera position (28) of the camera through the image point (46) to the water surface (22) of the water area according to the position data and the pixel data. The method according to claim 12.

14. The positional relationship refers to the range (50) between the own ship and the target ship (34) and / or the orientation (AB, RB) of the target ship (34). The method according to claim 12 or 13.

15. The pixel data of the at least one image point (46) is determined such that the image point (46) corresponds to the real-world point (48) of the ship-water interface (40) closest to the own ship. The method according to any one of claims 12 to 14.

16. A system for determining the positional relationship between an own ship in a water area and a target ship (34) in the water area, a camera for capturing an image (20) around the own ship, the camera being arranged on the own ship, and a processing unit coupled to the camera and configured to execute the method according to any one of claims 12 to 15. A system comprising the above.

Images