Camera-based collision alerts for marine vessels

Abstract

A system comprising one or more processing circuitries configured to: obtain a plurality of images of at least one marine object, acquired by an imaging device of a marine vessel; use at least some of the plurality of images to determine: data Dbearing informative of a direction of the at least one marine object, and data Dimension informative of one or more dimensions of the at least one marine object in the at least some of the plurality of images; and to use data Dbeanng and data Dimension to determine at least one of: data informative of a collision risk between the marine vessel and the marine object, data informative of a closest point of approach between the marine vessel and the at least one marine object, or data informative of a time to the closest point of approach between the marine vessel and the at least one marine object.

Description

[0001] CAMERA-BASED COLLISION ALERTS FOR MARINE VESSELS

[0002] RELATED APPLICATIONS

[0003] This application claims priority from Israeli patent application serial number 312,561 filing date May 2, 2024.

[0004] TECHNICAL FIELD

[0005] The presently disclosed subject matter relates to the field of marine environment.

[0006] BACKGROUND

[0007] In a marine environment, a marine vessel travels on a route on which it can encounter various situations. Some of these situations can include dangers, e.g., an obstacle to be avoided, zones with dangerous weather, etc.

[0008] In order to enable control of the marine vessel (by the crew and / or by an automatic pilot), it is required to determine information on marine objects encountered by the marine vessel.

[0009] It is now necessary to provide new methods and systems in order to improve safety and reliability of marine vessels navigation, improve understanding of the marine environment for marine vessels, and improve control of marine vessels. More generally, it is necessary to develop innovative methods in the marine domain, and in particular, in the field of autonomous ships.

[0010] GENERAL DESCRIPTION

[0011] In accordance with certain aspects of the presently disclosed subject matter, there is provided a system comprising one or more processing circuitries configured to obtain a plurality of images of at least one marine object, acquired by an imaging device of a marine vessel, use at least some of the plurality of images to determine data Dbeanng informative of a direction of the at least one marine object, and data Dimension informative of one or more dimensions of the at least one marine object in the at least some of the plurality of images, and use data Dbeanng and data Ddimension to determine at least one of (a) data Dnsk informative of a collision risk between the marine vessel and the marine object, (b) data DCPA informative of a closest point of approach (CPA) between the marine vessel and the at least one marine object, or (c) data DTCPA informative of a time to the closest point of approach (TCP A) between the marine vessel and the at least one marine object.

[0012] In addition to the above features, the system according to this aspect of the presently disclosed subject matter can optionally comprise one or more of features (i) to (xxiii) below, in any technically possible combination or permutation: i. data Dbeanng is informative of a variation of the direction of the at least one marine object in the at least some of the plurality of images; ii . data Dbeanng is informative of a bearing angle variation of the marine obj ect in the at least some of the plurality of images; iii. data Ddimension is informative of a variation in the one or more dimensions of the at least one marine object in the at least some of the plurality of images; iv. the one or more processing circuitries are operative to implement at least one machine learning model, wherein the one or more processing circuitries are configured to feed data Dbeanng and data Ddimension to the at least one machine learning model to determine, by the at least one machine learning model, at least one of data Dnsk, data DCPA or data DTCPA; v. the at least one machine learning model has been trained with a training set including, for each given marine object of a plurality of marine objects, data Dhoaring informative of a direction of the given marine object in images of the given marine object acquired by an imaging device of a given marine vessel, data Ddimension informative of one or more dimensions of the given marine object in said images, at least one of data Dnsk informative of a collision risk between the given marine vessel and the given marine obj ect, data DCPA informative of a closest point of approach between the given marine vessel and the given marine object, or data DTCPA informative of a time to the closest point of approach between the given marine vessel and the given marine object; vi. the one or more processing circuitries are configured to obtain a model, modelling a relationship between data Dbeanng and data Ddimension of at least one given marine object, and at least one of data Dnsk informative of a collision risk between a given marine vessel and the at least one given marine object, or data DCPA informative of a closest point of approach between the given marine vessel and the at least one given marine object, or data DTCPA informative of a time to the closest point of approach between the given marine vessel and the given marine object; use the model, data Dbearing informative of a direction of the at least one marine object, and data Dimension informative of one or more dimensions of the at least one marine object, to determine at least one of: data Dnsk informative of a collision risk between the marine vessel and the at least one marine object, data DCPA informative of a closest point of approach between the marine vessel and the at least one marine object, or data DTCPA informative of a time to the closest point of approach between the marine vessel and the at least one marine object; vii. the system is configured to determine at least one of data Dnsk informative of a collision risk between the marine vessel and the at least one marine object, data DCPA informative of a closest point of approach between the marine vessel and the at least one marine object, or data DTCPA informative of a time to the closest point of approach between the marine vessel and the at least one marine object, without using data informative of the at least one marine object acquired by a sensor which is not a camera; viii. the system is configured to determine at least one of data Dnsk informative of a collision risk between the marine vessel and the at least one marine object, data DCPA informative of a closest point of approach between the marine vessel and the at least one marine object, or data DTCPA informative of a time to the closest point of approach between the marine vessel and the at least one marine object, without using data informative of the at least one marine object acquired by a radar system and an automatic identification system (AIS); ix. the system is configured to determine, for each given marine object of a plurality of marine objects present in the plurality of images, at least one of data Dnsk informative of a collision risk between the marine vessel and the given marine object, data DCPA informative of a closest point of approach between the marine vessel and the given marine object, or data DTCPA informative of a time to the closest point of approach between the marine vessel and the given marine object; x. the system is configured to use the at least some of the plurality of images to determine data Dbearing short, time and data Ddimension short time for the at least one marine object over a first period of time, and data Dbearing medium time and data Ddimension medium time for the at least one the marine obj ect over a second period of time, wherein the second period of time is longer than the first period of time, use data Dbearing short time, data Ddimension short time, data Dbearing medium time and data Ddimension medium time to determine at least one of data Dnsk informative of a collision risk between the marine vessel and the at least one marine object, data DCPA informative of a closest point of approach between the marine vessel and the at least one marine object, or data DTCPA informative of a time to the closest point of approach between the marine vessel and the at least one marine object; xi. the system is configured to feed data Dbearing short time, data Ddimension short time, data Dbearing medium time and data Ddimension medium time tO at least one trained machine learning model to determine, by the at least one trained machine learning model, at least one of data Dnsk informative of a collision risk between the marine vessel and the at least one marine object, data DCPA informative of a closest point of approach between the marine vessel and the at least one marine object, or data DTCPA informative of a time to the closest point of approach between the marine vessel and the at least one marine object; xii. the system is configured to use the at least some of the plurality of images to determine data Dbearing short, time and data Ddimension short time for the at least one marine object over a first period of time, and data Dbearing medium time and data Ddimension medium time for the at least one marine object over a second period of time, wherein the second period of time is longer than the first period of time, perform at least one of using data Dbearing short time and data Ddimension short time to determine first data Dnsk i informative of a collision risk between the marine vessel and the at least one marine object and using data Dbearing medium time and data Ddimension medium time to determine second data Dnsk 2 informative of a collision risk between the marine vessel and the at least one marine object; using data Dbearing short time and data Ddimension short time to determine first data DCPA i informative of a closest point of approach between the marine vessel and the at least one marine object and using data Dbearing medium time and data D dimension _medium _time tO determine second data DCPA 2 informative of a closest point of approach between the marine vessel and the at least one marine object; or using data Dbearing short time and data Ddimension _short_time to determine first data DTCPA 1 informative of a time to the closest point of approach between the marine vessel and the at least one marine object and using data Dbearing medium time and data Ddimension medium time to determine second data DTCPA 2 informative of a time to the closest point of approach between the marine vessel and the at least one marine object; xiii. the system is configured to aggregate the first data Dnsk 1 informative of a collision risk between the marine vessel and the at least one marine object and the second data Dnsk 2 informative of a collision risk between the marine vessel and the at least one marine object to determine data Dnsk informative of a collision risk between the marine vessel and the at least one marine object; xiv. the system is configured to aggregate the first data DCPA 1 informative of a closest point of approach between the marine vessel and the at least one marine object and the second data DCPA 2 informative of a closest point of approach between the marine vessel and the at least one marine object to determine data DCPA informative of a closest point of approach between the marine vessel and the at least one marine object; xv. the system is configured to use data and data Ddimension short time to determine first data DTCPA 1 informative of a time to the closest point of approach between the marine vessel and the at least one marine object and using data Dbearing medium time and data Ddimension medium time to determine second data DTCPA 2 informative of a time to the closest point of approach between the marine vessel and the at least one marine object; xvi. the one or more processing circuitries are operative to implement a first machine learning model and a second machine learning model different from the first machine learning model, wherein the system is configured to use the at least some of the plurality of images to determine data Dbearing short time and data Ddimension short time for the at least one marine object over a first period of time, and data Dbearing medium time and data D dimension medium time for the at least one marine object over a second period of time, wherein the second period of time is longer than the first period of time, perform at least one of: feeding data Dbearing shoitjime and data Ddimension short time to the first machine learning model to determine first data Dnsk i informative of a collision risk between the marine vessel and the at least one marine object and feeding data Dbearing medium time and data to the second machine learning model to determine second data Dnsk 2 informative of a collision risk between the marine vessel and the at least one marine object; feeding data Dbearing shmt time and data Ddimension shmt time to the first machine learning model to determine first data DCPA 1 informative of a closest point of approach between the marine vessel and the at least one marine object and feeding data Dbearing medium time and data Ddimension medium time to the second machine learning model to determine second data DCPA 2 informative of a closest point of approach between the marine vessel and the at least one marine object; or feeding data Dbearing short time and data Ddimension short time to the first machine learning model to determine first data DTCPA 1 informative of a time to the closest point of approach between the marine vessel and the at least one marine obj ect and feeding data Dbearing medium time and data Ddimension medium time to the second machine learning model to determine second data DTCPA 2 informative of a time to the closest point of approach between the marine vessel and the at least one marine object; xvii. the first machine learning model has been trained using data Dbearing and data Ddimension determined for one or more marine objects based on images collected over periods of time matching the first period of time according to a criterion; xviii . the second machine learning model has been trained using data Dbearing and data Ddimension determined for one or more marine objects based on images collected over periods of time matching the second period of time according to a criterion; xix. the system is configured to determine at least one of data Dnsk informative of a collision risk between the marine vessel and the at least one marine object, data DCPA informative of a closest point of approach (CPA) between the marine vessel and the at least one marine object, or data DTCPA informative of a time to the closest point of approach (TCP A) between the marine vessel and the at least one marine object, without estimating a distance between the marine vessel and the at least one marine object; xx. the system is configured to configured to transmit at least one of data Dusk informative of a collision risk between the marine vessel and the at least one marine object, data DCPA informative of a closest point of approach (CPA) between the marine vessel and the at least one marine object, or data DTCPA informative of a time to the closest point of approach (TCP A) between the marine vessel and the at least one marine obj ect to a computer- implemented controller operative to control at least one of a position, a direction, a velocity, or an acceleration of the marine vessel; xxi . the controller is operative to use at least one of data Dnsk, data DCPA or data DTCPA to modify at least one of at least one of a position, a direction, a velocity or an acceleration of the marine vessel, to avoid a collision or a near-collision with the at least one marine object; xxii. the system is configured to display one or more images of the plurality of images of the at least one marine object, acquired by the imaging device of the marine vessel, informative data superimposed on said on or more images, the informative data including at least one of data Dnsk informative of a collision risk between the marine vessel and the at least one marine object, data DCPA informative of a closest point of approach (CPA) between the marine vessel and the at least one marine object, or data DTCPA informative of a time to the closest point of approach (TCP A) between the marine vessel and the at least one marine object; and xxiii . the system is configured to repetitively recalculate data Dbearing informative of a direction of the at least one marine object, and data Dimension informative of one or more dimensions of the at least one marine object over time, and to update an estimate of data Dnsk informative of a collision risk between the marine vessel and the at least one marine obj ect, data DCPA informative of a closest point of approach (CPA) between the marine vessel and the at least one marine object, or data DTCPA informative of a time to the closest point of approach between the marine vessel and the at least one marine object. In accordance with other aspects of the presently disclosed subject matter, there is provided a marine vessel comprising an imaging device, and one or more processing circuitries configured to obtain a plurality of images of at least one marine object, acquired by the imaging device of the marine vessel, use at least some of the plurality of images to determine data Dbeanng informative of a direction of the at least one marine object, and data Dimension informative of one or more dimensions of the marine object in the at least some of the plurality of images, and use data Dbeanng and data Ddimension to determine at least one of data Dnsk informative of a collision risk between the marine vessel and the at least one marine object, data DCPA informative of a closest point of approach (CPA) between the marine vessel and the at least one marine object, or data DTCPA informative of a time to the closest point of approach (TCP A) between the marine vessel and the at least one marine object.

[0013] In addition to the above features, the marine vessel according to this aspect of the presently disclosed subject matter can optionally comprise one or more of features (i) to (xxiii) above, in any technically possible combination or permutation.

[0014] In accordance with other aspects of the presently disclosed subject matter, there is provided a system comprising one or more processing circuitries configured to implement at least one machine learning model, wherein the one or more processing circuitries are configured to obtain a plurality of images of at least one marine object, acquired by an imaging device of a marine vessel, feed at least some of the plurality of images to the machine learning model to determine at least one of (a) data Dnsk informative of a collision risk between the marine vessel and the at least one marine object, (b) data DCPA informative of a closest point of approach (CPA) between the marine vessel and the at least one marine object, or (c) data DTCPA informative of a time to the closest point of approach (TCP A) between the marine vessel and the at least one marine object.

[0015] In addition to the above features, the system according to this aspect of the presently disclosed subject matter can optionally comprise one or more of features (xxiv) to (xxv) below, in any technically possible combination or permutation: xxiv. the system is configured to, for each image of the at least some of the plurality of images, determine an area in which the at least one marine object is located, and feed the area to the machine learning model; xxv. the machine learning model has been trained using a training set including, for each given marine object of a plurality of marine objects, images of the given marine object acquired by an imaging device of a given marine vessel, at least one of (i) data Dusk informative of a collision risk between the given marine vessel and the given marine object; (ii) data DCPA informative of a closest point of approach (CPA) between the given marine vessel and the given marine object, or (iii) data DTCPA informative of a time to the closest point of approach (TCP A) between the given marine vessel and the given marine object.

[0016] In accordance with other aspects of the presently disclosed subject matter, there is provided a controller of a marine vessel, the controller comprising one or more processing circuitries configured to: obtain at least one of data Dusk informative of a collision risk between the marine vessel and at least one marine object, determined based on data Dbearing informative of a direction of the at least one marine object in images acquired by an imaging device of the marine vessel, and data Dimension informative of one or more dimensions of the at least one marine object in said images, data DCPA informative of a closest point of approach (CPA) between the marine vessel and the at least one marine object, determined based on data Dbearing informative of a direction of the at least one marine object in images acquired by an imaging device of the marine vessel, and data Dimension informative of one or more dimensions of the at least one marine object in said images, or data DTCPA informative of a time to the closest point of approach (TCP A) between the marine vessel and the at least one marine object, determined based on data Dbearing informative of a direction of the at least one marine object in images acquired by an imaging device of the marine vessel, and data Dimension informative of one or more dimensions of the at least one marine object in said images, use at least one of the data Dnsk, the data DCPA or the data DTCPA to control at least one of a position, a velocity, or an acceleration of the marine vessel to avoid a collision with the at least one marine object.

[0017] In accordance with other aspects of the presently disclosed subject matter, there is provided a method comprising, by one or more processing circuitries, obtaining a plurality of images of at least one marine object, acquired by an imaging device of a marine vessel, using at least some of the plurality of images to determine data Dbearing informative of a direction of the at least one marine object, and data Dimension informative of one or more dimensions of the at least one marine object in the at least some of the plurality of images, and using data Dbearing and data Dimension to determine at least one of (a) data Dnsk informative of a collision risk between the marine vessel and the at least one marine obj ect, (b) data DCPA informative of a closest point of approach (CPA) between the marine vessel and the at least one marine object, or (c) data DTCPA informative of a time to the closest point of approach (TCP A) between the marine vessel and the at least one marine object.

[0018] In addition to the above features, the method according to this aspect of the presently disclosed subject matter can optionally comprise one or more of features (i) to (xxiii) above (described with respect to the system), in any technically possible combination or permutation.

[0019] In accordance with other aspects of the presently disclosed subject matter, there is provided a non-transitory computer readable medium comprising instructions that, when executed by one or more processing circuitries, cause the one or more processing circuitries to perform this method.

[0020] In accordance with other aspects of the presently disclosed subject matter, there is provided a method comprising, by one or more processing circuitries, obtaining a plurality of images of at least one marine object, acquired by an imaging device of a marine vessel, feeding at least some of the plurality of images to the machine learning model to determine at least one of (a) data Dnsk informative of a collision risk between the marine vessel and the at least one marine object, (b) data DCPA informative of a closest point of approach (CPA) between the marine vessel and the at least one marine object, or (c) data DTCPA informative of a time to the closest point of approach (TCP A) between the marine vessel and the at least one marine object.

[0021] In addition to the above features, the method according to this aspect of the presently disclosed subject matter can optionally comprise or more of features (xxiv) to (xxv) above (described with respect to the system), in any technically possible combination or permutation.

[0022] In accordance with other aspects of the presently disclosed subject matter, there is provided a non-transitory computer readable medium comprising instructions that, when executed by one or more processing circuitries, cause the one or more processing circuitries to perform this method.

[0023] According to some embodiments, the proposed solution enables estimating a collision risk between a marine vessel and one or more marine objects in an accurate and efficient way. According to some embodiments, the proposed solution enables estimating a collision risk between a marine vessel and one or more marine objects using only images provided by an imaging device.

[0024] According to some embodiments, the proposed solution enables estimating a collision risk between a marine vessel and one or more marine objects without requiring data of non-imaging sensors, such a radar system and / or an automatic identification system (AIS).

[0025] According to some embodiments, the proposed solution enables estimating a collision risk between a marine vessel and one or more marine objects using a single camera.

[0026] According to some embodiments, the proposed solution dynamically estimates a collision risk between a marine vessel and one or more marine objects.

[0027] According to some embodiments, the proposed solution is usable to ensure automatic control of a route of a marine vessel. In particular, it can be used for autonomous ships.

[0028] According to some embodiments, the proposed solution enables determining a collision risk in various possible scenarios, such as congested water, open sea, etc.

[0029] According to some embodiments, the proposed solution enables to determine data informative of the closest point of approach (CPA), and / or of the time to the closest point of approach (TCP A).

[0030] According to some embodiments, the proposed solution enables determining a collision risk in real time, or quasi real time.

[0031] According to some embodiments, the proposed solution enables determining a collision risk using multiple cameras.

[0032] According to some embodiments, the proposed solution can be easily deployed on a marine vessel.

[0033] According to some embodiments, the proposed solution is not limited by the physical limitation (range limitation) associated with a direct estimation of a distance to the marine object(s).

[0034] According to some embodiments, the proposed solution enables determining a risk of collision (or other data informative of a potential collision) with marine objects, without requiring an explicit determination of the distance to the marine objects. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to understand the invention and to see how it can be carried out in practice, embodiments will be described, by way of non-limiting examples, with reference to the accompanying drawings, in which:

[0036] - Fig- 1 illustrates a system which can be used to perform one or more of the methods described hereinafter, according to some examples of the invention;

[0037] Fig. 2A illustrates a method of determining various data informative of a collision risk between a marine vessel and one or more marine objects, according to some examples of the invention;

[0038] Fig. 2B illustrates an example of an image of marine objects acquired by the imaging device of a marine vessel;

[0039] Fig. 2C illustrates a schematic representation of a variation in the bearing angle of a marine object over time;

[0040] - Fig. 2D illustrates a bearing angle of a marine object;

[0041] Fig. 2E illustrates data which can be used to determine the bearing angle of a marine object, based on images of the marine object;

[0042] Fig. 2F illustrates an example of tracking of marine objects in different images; Fig. 3A illustrates a method of displaying various data informative of a collision risk between a marine vessel and one or more marine objects, according to some examples of the invention;

[0043] Fig. 3B illustrates an example of the method of Fig. 3A;

[0044] - Fig. 4 illustrates a non-limitative example of variations in the bearing angle of a marine object, which leads to collision;

[0045] - Fig. 5 illustrates another method of determining various data informative of a collision risk between a marine vessel and one or more marine objects, according to some examples of the invention;

[0046] Fig. 6A illustrates different periods of time which can be used to compute data in the method of Fig. 5;

[0047] Figs. 6B to 6D illustrate non-limitative examples of architectures which can be used to implement the method of Fig. 5;

[0048] - Fig. 7 illustrates another method of determining various data informative of a collision risk between a marine vessel and one or more marine objects, according to some examples of the invention; - Fig. 8 illustrates a method of using various data informative of a collision risk between a marine vessel and one or more marine objects to control the marine vessel, according to some examples of the invention;

[0049] - Fig- 9 illustrates a method of training a machine learning model usable to determine various data informative of a collision risk between a marine vessel and one or more marine objects, according to some examples of the invention; and

[0050] Fig. 10 illustrates another method of training a machine learning model usable to determine various data informative of a collision risk between a marine vessel and one or more marine objects, according to some examples of the invention.

[0051] DETAILED DESCRIPTION

[0052] Attention is now drawn to Fig. 1, which depicts a system 100 usable in the marine environment, in particular to determine various data informative of a collision risk between a marine vessel 125 and one or more marine objects. At least part of the system 100 is usable to execute the methods described with reference to Figs. 2A, 3A, 5 and 7 to 10.

[0053] The marine objects can include e.g., other marine vessels, icebergs, buoys, etc. The marine objects generally include at least a part which is located above sea level.

[0054] Elements of the system 100 depicted in Fig. 1 can be made up of any combination of software and hardware and / or firmware. Elements of the system 100 depicted in Fig. 1 may be centralized in one location or dispersed over more than one location. In other examples of the presently disclosed subject matter, the system of Fig. 1 may comprise fewer, more, and / or different elements than those shown in Fig. 1. Likewise, the specific division of the functionality of the disclosed system to specific parts, as described below, is provided by way of example, and other various alternatives are also construed within the scope of the presently disclosed subject matter.

[0055] System 100 includes a computerized system 101. The computerized system 101 includes one or more processing circuitries 110, which each includes one or more processors and one or more memories.

[0056] According to some examples, the computerized system 101 is embedded on a marine vessel 125. Marine vessels include e.g., ships, boats, hovercrafts, etc. Note that this is not limitative, and, in some examples, the computerized system 101 can be located on ground (or on a different marine vessel) and communicate with sensor(s) 130 of the marine vessel 125.

[0057] As shown in Fig. 1, the computerized system 101 can obtain data from one or more sensors 130. At least some of the sensors 130 can be located on the marine vessel 125 (or on at least one marine object communicating with the marine vessel 125).

[0058] Sensors 130 collect data during the voyage of the marine vessel 125. The voyage includes portions of the voyage in which the marine vessel 125 is in motion but can also include portions of the voyage in which the marine vessel 125 is substantially static (e.g., when the marine vessel 125 is moored or docked, such as in a harbor).

[0059] Sensors 130 include an imaging device 120 (e.g., a camera), which is mounted on the marine vessel 125. In some examples, the camera includes an infrared camera, a night camera, a day camera, etc.

[0060] In some examples, sensors 130 include a plurality of imaging devices 120 (which can be distinct). In some examples, the imaging devices 120 may have different fields of view (which do not overlap at all) or may have a field of view which can at least partially overlap.

[0061] According to some embodiments, sensors 130 include one or more additional sensors 115 (which are not necessarily imaging devices) such as (this list is not limitative) a radar (any type of radar), a LIDAR, an automatic identification system (AIS) (located on the marine vessel 125 and / or on a marine object distinct from the marine vessel 125), a transponder communicating with a GPS located on other marine object(s), a system which includes a laser located on the marine vessel 125, and an optical reflector located on another marine object to be located by the marine vessel 125 (reflection of the laser by the reflector enables localization of the other marine object), etc.

[0062] In particular, sensors 115 provide information usable to localize marine objects surrounding the marine vessel 125.

[0063] The marine vessel 125 itself can include other sensors, such a geo-localization system (e.g., GPS), IMU, velocity and acceleration sensors, a gyro compass, etc.

[0064] As explained hereinafter in the specification, the computerized system 101 can process data collected by one or more of the sensors 130.

[0065] In some examples, data output by the computerized system 101 can be transmitted through a remote communication network 140 towards e.g., a central station 150, which can include one or more processing circuitries 161. In some embodiments, the central station 150 can perform at least some of the tasks of the one or more processing circuitries 110.

[0066] The remote communication network 140 can correspond e.g., to a broadband cellular network (e.g. 4G, 5G, LTE, etc.), a satellite communication network, radio communication network (such as Radio VHF - very high frequency), etc.

[0067] Data can be transmitted using a communication system (not represented) located on the marine vessel 125, which is suitable to transmit data via the remote communication network 140. The communication system can include e.g., an antenna, an emitter, a transponder, etc.

[0068] In some examples, data output by the computerized system 101 can be transmitted to a controller 131 of the marine vessel 125. The controller 131 is a computer-implemented controller which can control the route of the marine vessel 125. Typically, the controller 131 can generate commands for actuators of the marine vessel 125, in order to control position and / or velocity and / or acceleration and / or direction of the marine vessel 125.

[0069] As visible in Fig. 1, in some examples, the one or more processing circuitries 110 can be configured to implement at least one first machine learning model 160. As explained hereinafter, the first machine learning model 160 can be trained to receive data extracted from images of one or more marine objects (in particular, data informative of variations in their direction and dimension(s) in the images), acquired by the imaging device 120 of the marine vessel 125, in order to generate data informative of a potential collision between the marine vessel 125 and the one or more marine objects.

[0070] In some examples, the one or more processing circuitries 110 can be configured to implement at least one second machine learning model 162. As explained hereinafter, the second machine learning model 162 can be trained to receive images of one or more marine objects acquired by the imaging device 120 of the marine vessel 125, in order to generate data informative of a potential collision between the marine vessel 125 and the one or more marine objects.

[0071] In some examples, the one or more processing circuitries 110 can be configured to implement at least one image processing algorithm 159, usable to detect marine objects in the images acquired by the imaging device 120 of the marine vessel 125. The image processing algorithm 159 can implement also a machine learning model (third machine learning model), but this is not mandatory. Although Fig. 1 depicts the first, second, and third machine learning models as distinct models, in some examples it is possible to use a common machine learning model for at least some of these machine learning models.

[0072] In some examples, the first machine learning model 160 (respectively, second and / or third machine learning model) can include a neural network (NN). In some embodiments, the first machine learning model 160 (respectively, second and / or third machine learning model) can include a deep neural network (DNN).

[0073] In particular, the processor can execute several computer-readable instructions implemented on a computer-readable memory comprised in the one or more processing circuitries 110, wherein execution of the computer-readable instructions enables data processing by the first machine learning model 160 (respectively second or third machine learning model).

[0074] By way of non-limiting example, the layers of the first machine learning model 160 (respectively second or third machine learning model) can be organized in accordance with Convolutional Neural Network (CNN) architecture, Recurrent Neural Network architecture, Recursive Neural Networks architecture, Generative Adversarial Network (GAN) architecture, or otherwise. In some embodiments, at least some of the layers can be organized in a plurality of DNN sub-networks. Each layer of the DNN can include multiple basic computational elements (CE), typically referred to in the art as dimensions, neurons, or nodes.

[0075] Generally, computational elements of a given layer can be connected with CEs of a preceding layer and / or a subsequent layer. Each connection between a CE of a preceding layer and a CE of a subsequent layer is associated with a weighting value. A given CE can receive inputs from CEs of a previous layer via the respective connections, each given connection being associated with a weighting value which can be applied to the input of the given connection. The weighting values can determine the relative strength of the connections and thus the relative influence of the respective inputs on the output of the given CE. The given CE can be configured to compute an activation value (e.g., the weighted sum of the inputs) and further derive an output by applying an activation function to the computed activation. The activation function can be, for example, an identity function, a deterministic function (e.g., linear, sigmoid, threshold, or the like), a stochastic function, or other suitable function. The output from the given CE can be transmitted to CEs of a subsequent layer via the respective connections. Likewise, as above, each connection at the output of a CE can be associated with a weighting value which can be applied to the output of the CE prior to being received as an input of a CE of a subsequent layer. Further to the weighting values, there can be threshold values (including limiting functions) associated with the connections and CEs.

[0076] Attention is now drawn to Fig. 2 A.

[0077] The method of Fig. 2A includes obtaining (operation 200) a plurality of images of at least one marine object, acquired by the imaging device 120 of the marine vessel 125. The plurality of images has been acquired by the imaging device 120 over time, during the voyage of the marine vessel 125. Note that in some examples, at least one (or more) of the plurality of images can include a plurality of marine objects.

[0078] A non-limitative example is illustrated in Fig. 2B, in which the imaging device 120 (not visible in Fig. 2B) mounted on the marine vessel 125 has acquired an image 280 of a plurality of marine objects 270i, 2702, 2703, 2704 and 270s.

[0079] As visible in the non-limitative example of Fig. 2B, an area (also called bounding box - see references 280i to 280s) indicates the (estimated) position of each marine object (e.g., 270i to 270s) in the image 280.

[0080] In order to determine the position of each marine object in the image, in some embodiments, one or more processing circuitries (such the one or more processing circuitries 110) can execute an algorithm which is configured to detect, in each image acquired by the imaging device 120, marine objects present in the image. In some examples, this algorithm enables tracking each marine object in the plurality of images.

[0081] Detection of marine object(s) in images can rely on the image processing algorithm 159 (see Fig. 1). As explained with reference to Fig. 1, in some examples the image processing algorithm 159 includes a third machine learning model (e.g., a neural network, such as a deep neural network) trained to detect marine objects present in images acquired by an imaging device of a marine vessel.

[0082] The output of identification of the marine object(s) in the image by the third machine learning model can include a geometric representation (e.g., a bounding box) indicative of the estimated location of each marine object(s) in the image.

[0083] Training of the third machine learning model can include supervised learning, in which a plurality of annotated images comprising marine objects are fed to the third machine learning model. Each annotated image can include e.g., an area / bounding box provided by an operator, who indicates the location of the marine object(s) in the image. This is not limitative, and the training can also include automatic training and / or non-supervised learning.

[0084] In some examples, the third machine learning model can provide information on the type of the object (e.g., marine vessel, type of marine vessel, type of marine object, such as an iceberg, etc.). This can be obtained by training the third machine learning model with a training method involving supervised learning, in which images comprising marine objects, and labels indicative of the positions and the types of the marine objects, are fed to the third machine learning model for its training.

[0085] In some examples, the first machine learning model 160 can be trained to identify, by itself, the marine object(s) in the image(s) received from the imaging device 120 (or from another imaging device).

[0086] The method of Fig. 2A further includes determining (operation 200) data (noted hereinafter data Dbearing) informative of a direction of at least one marine object in the plurality of images. In particular, for each image, data informative of the direction of the at least one marine object can be determined. As a consequence, data informative of the direction of the at least one marine object overtime can be determined. In some examples, variation(s) of the direction of the at least one marine object over time can be determined. In some examples, if the current image corresponds to time ti, data Dbearing associated with this current image can be informative of the difference between the bearing angle of the marine object at time ti and the bearing angle of the marine object in a previous image at time ti-i, or at time tj with j<i.

[0087] According to some examples, the direction of the marine object can be expressed as an angular position of the marine object in each image of the plurality of images. In particular, data Dbearing can be informative of a bearing angle of the marine object in each image of the plurality of images. In particular, the relative bearing angle of the marine object can be determined. Relative bearing angle refers to the angle between the forward direction (see reference 285 in Fig. 2C) of the marine vessel 125 and the location of the marine object 270. For example, an object relative bearing of 0 degrees would be directly ahead, and an object relative bearing of 180 degrees would be behind. Note that this is not limitative, and the direction of the marine object can be determined with respect to a different reference direction. In the example of Fig. 2C, the bearing angle of the marine object 270 at time to is equal to / ?toand the bearing angle of the marine object 270 at time to+At is equal to / ?t0+At-

[0088] Note that the bearing angle can be determined from the image as follows (this method is not limitative). Assume that a given marine object 299 (depicted in Fig. 2D) has been identified in an image acquired by the imaging device 120 of the marine vessel 125

[0089] For example, a bounding box (see Fig. 2E, “target Jjounding box") including the given marine object 299 is obtained. Assume the following notations: i. (Jef tupperx, lef^uppery) correspond to the coordinates of the left upper point of the bounding box (in the image); ii. (rightbottomx, rightbottomy) correspond to the coordinates of the right lower point of the bounding box (in the image); iii. camhorfovcorresponds to the imaging device horizontal field of view; iv. imgwcorresponds to the image resolution (width by height); v. camyawcorresponds to the yaw of the imaging device 120 (it is generally assumed that the imaging device 120 and the marine vessel 125 form a rigid body); vi . onimgbearingcorresponds to the bearing angle of the given marine obj ect 299 in the referential of the image; vii. globalbearingcorresponds to the bearing angle of the given marine object 299 in an absolute referential.

[0090] The coordinates of the two extremities of the bounding box can be converted into a single point, with coordinates (targetx, largely).

[0091] The coordinates (largely largely) of the given marine object 299 can then be converted into a bearing (noted g lob al _b earing) of the given marine object expressed in an absolute referential (e.g. Earth referential), as detailed hereinafter. cam_hor_fov leftimgangie= ?

[0092] The bearing of the given marine object 299 used in the data Dbearing can correspond to onimgbearlngor to globalbearing.

[0093] The method of Fig. 2A further includes determining (operation 220) data Dimension informative of one or more dimensions of the marine object in at least some (or all) of the plurality of images.

[0094] According to some examples, data Dimension can be determined for each image of the plurality of images.

[0095] According to some examples, determining data Dimension can include determining a size (pixel size) of the marine object in each image of the plurality of images.

[0096] In some examples, this can include measuring dimensions (e.g., height and / or width in pixels) of an area (bounding box) which indicates the position of the marine object in the image.

[0097] In some examples, this can include measuring the area (in pixels) of an area (bounding box) which encircles the marine object in the image.

[0098] According to some examples, data Dimension is informative of a change in the one or more dimensions of the marine object in the plurality of images (scale factor). In particular, for each image, a ratio between the size of the marine object in the current image, and the size of the marine object in the previous image (or in other past images) can be determined and used as data Dimension for the current image. If the current image corresponds to time ti, the previous image can correspond to time ti-i, or to time tj with j<i. The ratio between the size of the marine object in the image at time ti and the size of the marine object in the image at time ti-i, or at time tj with j<i, can be determined. This ratio can be computed for each image and stored as data Dimension of the marine object in each image.

[0099] Note that the ratio of change in the size of the marine object between two images depends on the distance between the imaging device (mounted on the marine vessel) and the marine object. As the marine object gets closer to the imaging device, the size of the marine object in the image increases (scale factor>l). Conversely, when the marine object gets further away from the imaging device, the size of the marine object in the image decreases (scale factor<l). In other words, Dimension provides information on the distance between the marine vessel and the marine object, without requiring an explicit determination of this distance.

[0100] According to some examples, the method further includes using (operation 230) data Dbearing and data Ddimension to determine data Dnsk informative of a collision risk between the marine vessel and the marine object.

[0101] According to some examples, data Dnsk includes a level of collision risk (such as high collision risk, medium collision risk, and low collision risk - note that a different number of risk levels can be used).

[0102] In some examples, this can include feeding data Dbearing and data Ddimension to the first machine learning model 160, to determine, by the first machine learning model 160, data Dnsk informative of a collision risk between the marine vessel 125 and the marine object. In particular, the first machine learning model 160 can be trained to generate, based on data Dbearing and data Ddimension, data Dnsk informative of a collision risk between the marine vessel 125 and the marine object. According to some examples, data Dnsk also includes confidence data, which indicates the level of confidence of the prediction (collision risk) output by the machine learning model.

[0103] In some examples, a model (which is not necessarily a machine learning model) has been built and links data Dbearing and data Ddimension to data Dnsk. This model can include e.g., a mathematical function which receives as an input data Dbearing and data Ddimension and provides as an output data Dnsk. This model can be built using training data. Assume that images of one or more marine objects have been obtained, which have been acquired by one or more imaging devices of one or more marine vessels. For each given marine object, the training data can include: data Dhoaring informative of a direction of the given marine obj ect, determined based on images of the given marine object acquired by an imaging device of a given marine vessel, data Ddimension informative of one or more dimensions of the marine object, determined based on these images, and a value for data Dnsk informative of a collision risk between the marine vessel and the marine object (data Dnsk can be provided e.g., by an operator, who can assign a level of risk for each scenario present in the training data).

[0104] A regression (or other interpolation / fitting method(s)) can be performed to create the model linking data Dbeanng and data Ddimension to data Dnsk.

[0105] According to some examples, operation 230 includes using data Dbeanng and data Dimension to determine data DCPA informative of a closest point of approach (CPA) between the marine vessel 125 and the marine object. Data DCPA can include a value corresponding to an estimate of the closest point of approach (CPA).

[0106] In some examples, this can include feeding data Dbeanng and data Ddimension to the first machine learning model 160, to determine, by the first machine learning model 160, data DCPA informative of a closest point of approach (CPA) between the marine vessel and the marine object. In particular, the first machine learning model 160 can be trained to generate, based on data Dbeanng and data Ddimension, data DCPA informative of a closest point of approach (CPA) between the marine vessel and the marine object.

[0107] In some examples, a model (which is not necessarily a machine learning model) has been built and links data Dbearing and data Ddimension to data DCPA. This model can include e.g., a mathematical function which receives as an input data Dbearing and data Ddimension and provides as an output data DCPA. This model can be built using training data. Assume that images of one or more marine objects have been obtained, which have been acquired by one or more imaging devices of one or more marine vessels. For each given marine object, the training data can include: data Dbeanng informative of a direction of the given marine obj ect, determined based on images of the given marine object acquired by an imaging device of a given marine vessel, data Ddimension informative of one or more dimensions of the marine object, determined based on these images, and a value for data DCPA informative of a closest point of approach (CPA) between the given marine vessel and the given marine object (data DCPA can be determined using the known positions of the given marine vessel and the given marine object over time, which can be determined using AIS and / or position sensors).

[0108] A regression (or other interpolation / fitting method(s)) can be performed to create the model linking data Dbeanng and data Ddimension to data DCPA. According to some examples, operation 230 includes using data Dbearing and data Dimension to determine data DTCPA informative of a time to the closest point of approach (TCP A) between the marine vessel and the marine object. Data DTCPA can include a value corresponding to an estimate of the time to the closest point of approach. Note that TCPA can be expressed as a remaining time (amount of time that remains up to the closest point of approach), and / or as an absolute time of the day at which the marine vessel will reach the closest point of approach or using other adapted convention(s).

[0109] In some examples, this can include feeding data Dbearing and data Dimension to the first machine learning model 160, to determine, by the first machine learning model 160, data DTCPA informative of a time to the closest point of approach between the marine vessel and the marine object. In particular, the first machine learning model 160 can be trained to generate, based on data Dbearing and data Ddimension, data DTCPA informative of a time to the closest point of approach (TCPA) between the marine vessel and the marine object.

[0110] In some examples, a model (which is not necessarily a machine learning model) has been built and links data Dbearing and data Ddimension to data DTCPA. This model can be implemented by the one or more processing circuitries 110. This model can include e.g., a mathematical function which receives as an input data Dbearing and data Ddimension and provides as an output data DTCPA. This model can be built using training data. Assume that images of one or more marine objects have been obtained, which have been acquired by one or more imaging devices of one or more marine vessels. For each given marine object, the training data can include: data Dhoaring informative of a direction of the given marine obj ect, determined based on images of the given marine object acquired by an imaging device of a given marine vessel, data Ddimension informative of one or more dimensions of the marine object, determined based on these images, and a value for data DTCPA informative of a time to the closest point of approach (TCPA) between the given marine vessel and the given marine object (data DTCPA can be determined using the known positions of the given marine vessel and the given marine object overtime, which can be determined using AIS and / or position sensors).

[0111] A regression (or other interpolation / fitting method(s)) can be performed to create the model linking data Dbearing and data Ddimension to data DTCPA. In some examples, a different machine learning model can be used to determine Dnsk, DCPA and DTCPA: a machine learning model can be used to determine Dnsk (this machine learning model has been trained to determine Dnsk based on data Dbeanng and data Dimension), another machine learning model can be used to determine DCPA (this machine learning model has been trained to determine DCPA based on data Dbeanng and data Ddimension), and another machine learning model can be used to determine DTCPA (this machine learning model has been trained to determine DTCPA based on data Dhearing and data Ddimension). In some examples, the same machine learning model can be used to determine Dnsk, DCPA, and DTCPA (in this case, the machine learning model has been trained to determine Dnsk, DCPA and DTCPA based on data Dhearing and data Ddimension). In some examples, a machine learning model can be used to determine Dnsk and another machine learning model can be used to determine DCPA and DTCPA. Examples of training methods of these one or more machine learning models are provided with reference to Fig. 9.

[0112] Note that the method of Fig. 2A can be repeated over time, with a certain periodicity, thereby enabling detecting the collision risk (and / or the CPA or TCP A) in real time or quasi real time. In some examples, each time a new image is obtained (or every N image, with N an integer which can be selected e.g., by a user, or which can be predefined), data Dbeanng and data Ddimension of the marine object is calculated for the current image and used to update data Dnsk and / or data DCPA and / or data DTCPA.

[0113] Note that the method of Fig. 2A can be performed in parallel for a plurality of marine objects. The plurality of marine objects can be located simultaneously in all, or at least in some of the plurality of images obtained from the imaging device 120. As a consequence, for each given marine object, at least one of data Dnsk informative of a collision risk between the marine vessel and the given marine object, data DCPA informative of a closest point of approach (CPA) between the marine vessel and the given marine obj ect, or data DTCPA informative of a time to the closest point of approach (TCP A) between the marine vessel and the given marine object, can be determined and output.

[0114] In some examples, each given marine object is tracked in the plurality of images in order to determine, in each image, data Dbeanng and data Ddimension of this given marine object.

[0115] A tracking method can be used to track each marine object over the different images. The tracking method can be implemented by one or more processing circuitries, such as the one or more processing circuitries 110. The tracking method can use e.g., a Kalman filter, or other adapted tracking methods can be used. The tracking method enables to understand the motion of each marine object over the plurality of images.

[0116] A non-limitative example of a tracking method is provided with reference to Fig. 2F

[0117] Assume that at time ti, an image is acquired by the imaging device 120. In this image, three marine objects are detected. A first marine object is located at position 230, a second marine object is located at position 231, and a third marine object is located at position 232.

[0118] At time t2 (different from ti), another image is acquired by the imaging device 120. In this image, three marine objects are detected. A first marine object is located at position 233, a second marine object is located at position 234, and a third marine object is located at position 235.

[0119] At time t3 (different from t2), another image is acquired by the imaging device 120. In this image, three marine objects are detected. A first marine object is located at position 236, a second marine object is located at position 237, and a third marine object is located at position 238.

[0120] A tracking method can be used to track the various marine objects over the different images. The tracking method can be implemented by one or more processing circuitries. The tracking method can implement e.g., a Kalman filter, or other adapted tracking methods.

[0121] In some embodiments, it can appear that a marine object is present in some of the images and disappears / is not present in subsequent images. This can be due to the relative motion between the marine object and the marine vessel 125.

[0122] In the example of Fig. 2F, the tracking method reveals that the marine object located at position 230 at time ti, the marine object located at position 233 at time t2 and the marine object located at position 236 at time t3 correspond to the same object at different periods of time. Therefore, the same tracking ID (in this example “(1)”) can be assigned to indicate that the same marine object is present at different positions in the different images.

[0123] Similarly, the tracking method reveals that the marine object located at position 231 at time ti, the marine object located at position 234 at time t2, and the marine object located at position 237 at time t3 correspond to the same marine object at different periods of time. Therefore, the same tracking ID (in this example “(2)”) can be assigned. Similarly, the tracking method reveals that the marine object located at position 232 at time ti, the marine object located at position 235 at time t2 and the marine object located at position 238 at time t3 correspond to the same marine object. Therefore, the same tracking ID (in this example “(3)”) can be assigned.

[0124] Attention is now drawn to Figs. 3A and 3B.

[0125] According to some examples, once data Dusk and / or data DCPA and / or data DTCPA has been determined for a given marine object (operation 300), at least some of the data Dnsk and / or data DCPA and / or data DTCPA can be displayed (e.g., to a user - see operation 310). In some embodiments, for each given marine object, the corresponding data Dnsk and / or data DCPA and / or data DTCPA is displayed to the user.

[0126] In some examples, the data Dnsk and / or data DCPA and / or data DTCPA is superimposed on the images acquired by the imaging device of the marine vessel. In particular, for each given image of the plurality of images, and for each given marine object present in the given image, data Dnsk and / or data DCPA and / or data DTCPA can be displayed on the given marine object (and / or in an area of the image associated with the given marine object, such as an area located in the vicinity of the given marine objet).

[0127] More generally, in some embodiments, informative data can be displayed on the images for each given marine object, such as at least one of the tracking ID, type of the given marine object, estimated velocity of the given marine object, estimated range or position of the given marine object, estimated bearing of the given marine object, duration for which the given marine object has been tracked, dimensions of the bounding box, etc. The informative data can also include Dnsk and / or data DCPA and / or data DTCPA.

[0128] A non-limitative example is illustrated in Fig. 3B, in which, for each marine object, tracking ID, duration for which the marine obj ect has been tracked, height of the bounding box, bearing angle, and level of collision risk (high, medium, low - corresponding to data Dnsk) are displayed.

[0129] This can be used e.g., by an operator of the marine vessel to control the route of the marine vessel. Note that the displayed data can be updated over time. These data can be also transmitted to a computerized system (such as an auto-pilot of the marine vessel) in order to control the route of the marine vessel.

[0130] Attention is now drawn to Fig. 4.

[0131] Fig. 4 provides a schematic explanation on the benefit of using data Dbeanng and data Ddimension to detect a collision risk between a marine vessel 400 and a marine object 410. In an “ideal” case, the bearing angle 420 of the marine object 410 remains constant as the range 430 of the marine object 410 (as mentioned above, data Dimension reflects the variations in range) decreases.

[0132] This indicates that the collision 450 will occur with certainty. In practice, various other patterns exist (with different variations of the bearing angle and / or of the range), which can correspond to various levels of collision risks (and, in turn, to various values of the CPA and / or TCP A). Therefore, it has to be understood that the scenario of Fig. 4 is not limitative and is provided as an example only.

[0133] Since data Dbeanng and data Dimension are beneficial to characterize the level of collision risk, these data can be used to determine data Dusk informative of a collision risk between the marine vessel and the marine object and / or data DCPA informative of a closest point of approach (CPA) between the marine vessel and the marine object and / or data DTCPA informative of a time to the closest point of approach between the marine vessel and the marine object.

[0134] Attention is now drawn to Figs. 5 and 6A to 6D.

[0135] The method of Fig. 5 includes obtaining (operation 500) a plurality of images (se reference 600 in Fig. 6A) of a marine object, acquired by an imaging device 120 of a marine vessel 125. Operation 500 is similar to operation 200 and is therefore not described again.

[0136] The method further includes using (operation 510) at least some of the plurality of images to determine data Dhpanng short time and data Dimension short time for the marine object over a first period of time (Ati). In particular, data Dbearing siwit time can be informative of variation(s) in the direction of the marine object over the first period of time, and data can be informative of variation(s) in one or more dimensions of the marine object in the at least some of the plurality of images, over the first period of time.

[0137] In some examples, data Dimension shoo time is informative of a variation in the one or more dimensions of the marine object between the beginning and the end of the first period of time.

[0138] In other examples, data Dimension short time is informative of a variation in the one or more dimensions of the marine object in each image collected within the first period of time (in other words, for each image collected within the first period of time, the difference between the one or more dimensions of the marine object in this image and the one or more dimensions of the marine object in the previous image is determined).

[0139] In some examples, data Dhparing is informative of a variation in the direction of the marine object between the beginning and the end of the first period of time. In other examples, data Dbeanng shoitjime is informative of a variation in the direction of the marine object in each image collected within the first period of time (in other words, for each image collected within the first period of time, the difference between the direction of the marine object in this image and the direction of the marine object in the previous image is determined).

[0140] The method further includes using (operation 520) at least some of the plurality of images to determine data Dbeanng medium time and data Ddimension _medium time for the marine object over a second period of time (At2), wherein the second period of time is longer than the first period of time. In particular, data Dearing medium time can be informative of variation(s) in the direction of the marine object over the second period of time, and data Ddimension medium time can be informative of variation(s) in one or more dimensions of the marine object in at least some of the plurality of images, over the second period of time.

[0141] In some examples, data Ddimension medium time is informative of a variation in the one or more dimensions of the marine object between the beginning and the end of the second period of time.

[0142] In other examples, data Ddimension medium time is informative of a variation in the one or more dimensions of the marine object in each image collected within the second period of time.

[0143] In some examples, data Dhearing medium time is informative of a variation in the direction of the marine object between the beginning and the end of the second period of time.

[0144] In other examples, data Dhearing medium time is informative of a variation in the direction of the marine object in each image collected within the second period of time.

[0145] In some examples, the first period of time and the second period of time can end at the same time, but this is not mandatory.

[0146] In some examples, the first period of time is included within the second period of time. For example, assume that a plurality of images has been acquired from time to to time t3. The first period of time can correspond to a period of time between time t2 and time t3, and the second period of time can correspond to a period of time between time ti and time t3, wherein to<ti<t2<t3.

[0147] The method further includes using (operation 530) data Dbeanng shoitjime, data Ddimension short time, data Dbeanng medium time and data Ddimension medium time to determine at least one of data Dnsk informative of a collision risk between the marine vessel and the marine object, data DCPA informative of a closest point of approach between the marine vessel and the marine object, or data DTCPA informative of a time to the closest point of approach between the marine vessel and the marine object.

[0148] Note that the method of Fig. 5 can be repeated over time. Each time a new image (or a plurality of new images) is acquired, the values of data Dbearing shmt time and data Ddimension shoit_time, can be recalculated based on this new image and past images collected over a period of time corresponding to the duration of the first period of time, and the values of data Dbearing incdiiiin time and data Ddimension medium time can be recalculated based on this new image and past images collected over a period of time corresponding to the duration of the second period of time. These new values for data Dbearing_shoit_time, data Ddimension short time, data Dbearing medium time and data Ddimension medium time can be used to update data Dnsk, data DCPA, or data DTCPA.

[0149] Note that operation 530 can be implemented in various ways.

[0150] In some examples, operation 530 includes feeding data Dheanng shmt time, data Ddimension short time, data Dbearing medium time and data Ddimension medium time to at least one model (which has been built to link data Dbearing and data Ddimension to at least one of data Dnsk and / or data DCPA and / or data DTCPA, as explained above), to determine data Dnsk and / or data DCPA and / or data DTCPA for the marine object.

[0151] In some examples, operation 530 includes feeding data Dbearing shoit time, data Ddimension short time, data Dbearing medium time and data Ddimension medium time to the first machine learning model 160, to determine data Dnsk and / or data DCPA and / or data DTCPA for the marine object.

[0152] Note that it is possible to use one machine learning model which receives data (data Ddimension and data Dbearing) informative of different periods of time, or to use a plurality of different machine learning models, each dedicated to receive data (data Ddimension and data Dbearing) informative of a different period of time (see e.g., Figs. 6B and 6C)

[0153] In some examples, operation 530 includes using data Dbearing shoit time and data Ddimension short time to determine a first estimate of data Dnsk and / or of data DCPA and / or of data DTCPA. This first estimate is determined using data Dbearing shon timp and data Dimension short time, which have been computed based on images collected within the first period of time. The method further includes using data Dbearing medium time and data Ddimension medium time to determine a second estimate of data Dnsk and / or data DCPA and / or data DTCPA. This second estimate is determined using data Dbearing medium time and data Ddimension medium time, which have been computed based on images collected within the second period of time. An aggregation of the first estimate and of the second estimate can be performed to determine respective aggregate values for data Dnsk and / or of data DCPA and / or of data DTCPA. The aggregation can include e.g., an average or other mathematical operations.

[0154] In some examples, the method can include using a different model for the “short term” prediction (corresponding to data collected within the first period of time) and for the “medium or long term” prediction (corresponding to data collected within the second period of time). A first model is fed with data Dbeanng short time and data Ddimension short time tO determine a first estimate of data Dnsk and / or of data DCPA and / or of data DTCPA. In some examples, the first model has been specifically built or trained for short term prediction, using data Dbeanng _short_time and data Ddimension short time computed for various marine objects based on images collected during short periods of time, which match the duration of the first period of time (or match the duration of the first period of time according to a criterion defining a maximal acceptable difference in their duration). The first model can correspond to a machine learning model or to a mathematical function linking an input (data Dbearing short time and data Ddimension _shoit time) to an output (data Dnsk and / or data DCPA and / or data DTCPA).

[0155] A second model can be fed with data Dbearing medium time and data Ddimension medium time to determine a second estimate of data Dnsk and / or of data DCPA and / or of data DTCPA. In some examples, the second model has been specifically built or trained for medium or long term prediction, using data Dbearing medium time and data Ddimension medium time computed for various marine objects based on images collected during “medium or long” periods of time, which match the duration of the second period of time (or match the duration of the second period of time according to a criterion defining a maximal acceptable difference in their duration). The second model can correspond to a machine learning model or to a mathematical function linking an input (data Dbearing medium time and data Ddimension medium time) to an output (data Dnsk and / or data DCPA and / or data DTCPA).

[0156] Note that the exact duration of the first period of time and the second period of time can be selected by an operator or can correspond to predefined values.

[0157] An example involving a plurality of machine learning models is described hereinafter. Note that this example can be performed using a plurality of models which are not machine learning models. A first machine learning model (see Fig. 6B) is fed with the data Dbeanng shoit_time and data Ddimension short time determined over the first period of time, and outputs data Dnsk i informative of a collision risk between the marine vessel and the marine object.

[0158] Note that the first machine learning model can be trained specifically with a training set including data Dbeanng and data Ddimension determined for marine objects based on images collected over a period of time matching the first period of time according to a criterion (e.g., with a difference below a threshold). The trained first machine learning model can then predict data Dnsk and / or data DCPA and / or data DTCPA for a given marine obj ect based on data Dbeanng and data Ddimension determined for a marine obj ect over a period of time matching the first period of time.

[0159] A second machine learning model is fed with the data Dbearing medium time and data D dimension medium time determined over the second period of time, and outputs data Dnsk 2 informative of a collision risk between the marine vessel and the marine object.

[0160] Note that the second machine learning model can be trained specifically with a training set including data Dbeanng and data Ddimension determined for marine objects based on images over a period of time matching the second period of time (e.g., with a difference below a threshold).

[0161] The method can include using data Dnsk 1 and data Dnsk 2 to determine aggregated data Dnsk informative of a collision risk between the marine vessel and the marine object. In a simple example, if each collision risk (data Dnsk 1 and data Dnsk 2) is scaled between 0 and 1, data Dnsk can be equal to the product of data Dnsk 1 and data Dnsk 2.

[0162] In some embodiments, a weighted aggregation can be performed, in which data Dnsk 1 is associated with a first weight and data Dnsk 2 is associated with a second weight.

[0163] In some embodiments, an additional machine learning model can be used to generate aggregated data Dnsk informative of a collision risk between the marine vessel and the marine object based on data Dnsk 1 and data Dnsk 2.

[0164] The additional machine learning model can be trained to perform this aggregation, using supervised learning.

[0165] Other types of aggregation can be performed, depending on the needs.

[0166] In some embodiments, it is not necessary to perform an aggregation, and both data Dnsk 1 and data Dnsk 2 are displayed to the user.

[0167] Note that the example above has been described with reference to the generation of data Dnsk 1 informative of a collision risk between the marine vessel and the marine obj ect (generated based on images collected during the first period of time), and data Dnsk 2 informative of a collision risk between the marine vessel and the marine object (generated based on data collected during the second period of time, which is longer than the first period of time).

[0168] The same method can be performed to generate data DCPA i informative of a closest point of approach (CPA) between the marine vessel and the marine object (data DCPA i corresponds to a first estimate of the CPA based on data collected during the first period of time) and data DCPA 2 informative of a closest point of approach (CPA) between the marine vessel and the marine object (data DCPA 2 corresponds to a second estimate of the CPA based on data collected during the second period of time, which is longer than the first period of time).

[0169] The same method can be performed to generate data DTCPA 1 informative of a time to the closest point of approach between the marine vessel and the marine object (data DTCOA 1 corresponds to a first estimate of the TCPA based on data collected during the first period of time) and data DTCPA 2 informative of a time to the closest point of approach between the marine vessel and the marine object (data DTCPA 2 corresponds to a second estimate of the TCPA based on data collected during the second period of time).

[0170] Note that the prediction of data Dusk and / or data DCPA and / or data DTCPA based on data collected during the first period of time (short period of time) has less sensitivity, but is more updated, whereas the prediction of data Dusk and / or data DCPA and / or data DTCPA based on data collected during the second period of time (longer period of time) is more sensitive, but less updated. Their aggregation enables improving the estimate of data Dusk and / or data DCPA and / or data DTCPA.

[0171] Note that the method of Fig. 5 can be generalized to a plurality of N periods of time Ati to AtN, for which Ati<At2<. . ,<AtN. In this case, for each period of time Ati (with l<i<N, wherein Ati corresponds to the duration of the period of time), data Dhr data Dimension- Ati are determined for the marine object, based on images corresponding to the period of time Ati. In some examples, assume that the current image corresponds to a current time tcurrent, then each period of time Ati can correspond to the period of time between “tcurrent-Ati” and “tcurrent”. It is therefore understood that the period of time Ati includes only the most recent images, whereas the period of time AtN is the period of time which includes the largest number of old images.

[0172] In some embodiments, each period of time Ati is included within the period of time Ati+i, but this is not mandatory. For example, Ati corresponds to the period of time between the current image and the image acquired one minute ago, whereas Ati+i corresponds to the period of time between the current image and the image acquired two minutes ago.

[0173] In some examples, all of the periods of time Att end at the same time, but this is not mandatory.

[0174] In some examples, data data Dimension Ati is informative of a variation in the one or more dimensions of the marine object between the beginning and the end of the period of time Ati.

[0175] In other examples, data Ddimension Ati is informative of a variation in the one or more dimensions of the marine object in each image collected within the period of time Ati (in other words, for each image collected within the period of time Ati, the difference between the one or more dimensions of the marine object in this image and the previous image is determined).

[0176] In some examples, data Dhp.-n-uH. Aii is informative of a variation in the direction of the marine object between the beginning and the end of the period of time Ati.

[0177] In other examples, data Dbearing Ati is informative of a variation in the direction of the marine object in each image collected within the period of time Ati (in other words, for each image collected within the period of time Ati, the difference between the bearing angle of the marine object in this image and the previous image is determined).

[0178] The method can include using data Dbearing Ati and data Ddimension Ati determined for a given marine object over each period of time Ati (with i from 1 to N) to determine data Dnsk and / or data DCPA and / or data DTCPA for the given marine object.

[0179] The method can include feeding data Pheanng Ati and data Ddimension Ati determined for a given marine object over each period of time Ati (with i from 1 to N) to a model, to determine data Dnsk and / or data DCPA and / or data DTCPA for the given marine object. The model can be the first machine learning model 160.

[0180] In some examples (see Fig. 6C), a plurality of N machine learning models MLito MLN is used (note that this applies similarly to N models which are not necessarily machine learning models). For a given marine object, each machine learning model MLi (with i from 1 to N) is fed with data Dbearing AH and data Ddimension AH determined for the given marine object over the period of time Ati.

[0181] Each machine learning model MLi outputs data Dnsk i informative of a collision risk between the marine vessel and the given marine object and / or data DCPA i informative of a closest point of approach (CPA) between the marine vessel and the given marine object and / or DTCPA i informative of a time to the closest point of approach between the marine vessel and the given marine object. The method can include using data Dnsk_i to data Dnsk N to determine aggregated data Dusk informative of a collision risk between the marine vessel and the given marine object, and / or using data DCPA i to data DCPA _N to determine aggregated data DCPA informative of a closest point of approach between the marine vessel and the given marine object and / or using data DTCPA i to data DTCPA _N to determine aggregated data DCPA informative of a time to the closest point of approach between the marine vessel and the given marine object.

[0182] In some examples, the aggregation includes using weights ai to ON, wherein each data Dnsk i (respectively each data DCPA i or each data DTCPA i) is associated with a corresponding weight ai.

[0183] In other examples, an additional machine learning model 660 can be used to generate aggregated data Dnsk informative of a collision risk between the marine vessel and the given marine object based on data Dnsk i to data Dnsk N (respectively aggregated data DCPA informative of a closest point of approach (CPA) between the marine vessel and the given marine obj ect, or aggregated data DTCPA informative of a time to the closest point of approach between the marine vessel and the given marine object).

[0184] In some examples, data Dnsk i to data Dnsk N (respectively data DCPA i to data DCPA N or data DTCPA i to data DTCPA N) are displayed to the user.

[0185] This method can be repeated overtime, with a certain periodicity, thereby enabling detecting the collision risk in real time or quasi real time. In some examples, each time a new image (or a plurality of N images) is obtained, data Dbearing and data Dimension of the marine object are recalculated over the different respective periods of time Ati using the current image, and are used to update the data Dnsk and / or the data DCPA and / or the data DTCPA.

[0186] As can be understood from the embodiments described above (see e.g., the methods described with reference to Figs. 2A and 5), the proposed solution enables determining data Dnsk informative of a collision risk between the marine vessel and the marine object and / or data DCPA informative of a closest point of approach and / or data DTCPA informative of a time to the closest point of approach, without using data informative of the marine object acquired by a sensor which is not an imaging device. In particular, it is possible to determine these data, without using data informative of the marine object acquired by a radar and / or an automatic identification system (AIS). This is beneficial, since not all marine objects embed an AIS. Similarly, a radar system is not always present on the marine vessel, and in some cases, the radar system is not operative to detect marine objects accurately.

[0187] In some examples, the proposed solution enables data Dusk informative of a collision risk between the marine vessel and the marine object and / or data DCPA informative of a closest point of approach and / or data DTCPA informative of a time to the closest point of approach using data provided only by one or more sensors, wherein all of the one or more sensors are imaging devices. In other words, the solution can rely, in some embodiments, only on the camera(s) mounted on the marine vessel, which is beneficial in many scenarios. In some examples, a single camera is sufficient to determine Dnsk and / or data DCPA and / or data DTCPA.

[0188] Note that this does not prevent the solution from receiving additional data from other sensors, if they are available, such as a radar system, AIS, etc. These additional data can be used e.g., to validate the collision risk or the estimated CPA / TCPA determined using the data extracted from the images of the marine object. However, as mentioned above, it is not mandatory to receive these additional data to determine the collision risk or the CPA / TCPA.

[0189] Attention is now drawn to Fig. 7, which depicts another method of determining data Dnsk and / or data DCPA and / or data DTCPA.

[0190] The method of Fig. 7 includes obtaining (operation 700) a plurality of images of at least one marine object, acquired by an imaging device of a marine vessel. Operation 700 is similar to operation 200 and is therefore not described again.

[0191] In some examples, the method of Fig. 7 can include determining, in each image, an area corresponding to the marine object in the image. In some embodiments, one or more processing circuitries (see e.g., 110) can execute an algorithm which is configured to detect, in each image acquired by the imaging device 120, marine objects present in the image. In some examples, this algorithm enables tracking each marine object in the plurality of images.

[0192] Detection of marine object(s) in images can rely on the image processing algorithm 159 (see Fig. 1). As explained with reference to Fig. 1, in some examples, the image processing algorithm 159 includes a third machine learning model (e.g., a neural network, such as a deep neural network) trained to detect marine objects present in images acquired by an imaging device of a marine vessel. The output of the identification of the marine object(s) in the image by the third machine learning model can include a geometric representation (e.g., a bounding box) indicative of the estimated location of each marine object(s) in the image.

[0193] The method of Fig. 7 further includes feeding at least some of the images and the areas (corresponding to the marine object) to a trained machine learning model (such as the second machine learning model 162) to determine at least one of: (i) data Dnsk informative of a collision risk between the marine vessel and the marine object and / or (ii) data DCPA informative of a closest point of approach between the marine vessel and the marine object and / or (iii) data DTCPA informative of a time to the closest point of approach (TCP A) between the marine vessel and the marine object.

[0194] Note that the method of Fig. 7 can be performed for a plurality of marine objects present in the plurality of images. In this case, the machine learning model is fed with the plurality of images, and, for each given marine object, with a plurality of areas in these images. The machine learning model outputs, for each given marine object, data Dnsk and / or data DCPA and / or data DTCPA.

[0195] The method of Fig. 7 differs from the method of Fig. 2A in that it is not necessary in the method of Fig. 7 to explicitly determine the attributes data Dbearing and Dimension. In some examples, operation 710 can be omitted and the second machine learning model 162 is also trained to determine the area in which each marine object is located in each image of the plurality of images.

[0196] Attention is now drawn to Fig. 8.

[0197] Once data Dnsk informative of a collision risk between the marine vessel and one or more marine objects, and / or data DCPA informative of a closest point of approach with one or more marine objects, and / or data DTCPA informative of a time to the closest point of approach with one or more marine objects, have been determined (operation 800), this can be used for controlling the marine vessel. In some embodiments, data Dnsk and / or data DCPA and / or data DTCPA can be output to an operator of the marine vessel (human operator), who can control the marine vessel accordingly. In particular, the operator can control the marine vessel to modify the direction (and / or the position and / or the velocity and / or the acceleration) of the marine vessel, to avoid a collision with the marine object(s). As mentioned above, data Dnsk and / or data DCPA and / or data DTCPA can be displayed on the images acquired by the imaging device.

[0198] In some embodiments, data Dnsk and / or data DCPA and / or data DTCPA can be transmitted (operation 810) to the controller 131 (computer-implemented controller) of the marine vessel. The controller 131 can correspond e.g., to the autopilot system of the marine vessel.

[0199] The controller 131 can send commands to actuators of the marine vessel, in order to modify at least one of the route and / or position and / or velocity and / or acceleration of the marine vessel, to avoid a collision with the marine object(s). The actuators can include the motor of the marine vessel, the steering device of the marine vessel, the various devices which enable the ship to change its trajectory, etc.

[0200] For example, the controller can be programmed, for a marine object which is associated with a medium or high risk of collision, to modify the route of the marine vessel, so that the bearing angle of the marine object increases (and / or the range to the marine object increases). In some embodiments, the controller can be programmed, for a marine object which is associated with a medium or high risk of collision, to reduce the velocity and / or acceleration of the marine vessel. This is however not limitative.

[0201] Note that operations 800 and 810 can be repeated over time. In particular, the risk of collision associated with each marine object can be dynamically updated over time. As a consequence, control of the route of the marine vessel by the controller can be updated dynamically over time. This facilitates obtaining an autonomous marine vessel, which embeds an automatic collision system avoidance.

[0202] Fig. 9 illustrates a method of training the first machine learning model 160.

[0203] The method includes obtaining (operation 900) a training set. The training set includes one or more sets of images Si to SN. Each set of images Si includes a plurality of images of one or more marine objects MOi,sito MOp,si acquired by an imaging device of a marine vessel. Note that the different sets of images can be acquired by imaging devices mounted on different marine vessels. This is however not mandatory.

[0204] For each given set of images Si, for each given marine object MJ.SI in the given set Si of images, data Dbearing informative of a direction of the given marine object MJ.SI in the given set of images Si and data Dimension informative of one or more dimensions of the marine object MJ.SI in the given set of images Si are determined (operation 910). In some examples, for each given marine object Mj.si, data Dbearing calculated for an image of the set Si can correspond to the variation in the bearing between this image and a previous image. Similarly, for each given marine obj ect MJ.SI, data Dimension calculated for an image of the set Si can correspond to the variation in the one or more dimensions of the given marine object between this image and a previous image. The method further includes associating (operation 920) each given marine object Mj,si in the given set Si of images with a label. The label can be indicative of at least one of: (i) a collision risk (e.g., high, medium, low, or any adapted label) between the marine vessel (on which the imaging device is located) and the marine object; (ii) a closest point of approach with the marine object; (iii) a time to the closest point of approach with the marine object.

[0205] Note that in some examples, for a given set Si of images acquired between time ti and time tM, it can occur that the level of collision risk evolves over time (because the route of the marine object and / or of the marine vessel can evolve), and therefore, the label may include a first collision risk for the period of time between ti and tK (with K<M) and a second collision risk for the period of time between tK and tM. This is however not mandatory.

[0206] In some examples, the collision risk of the label can be provided by an operator, who indicates the level of collision risk with the given marine object Mj.si in the given set Si of images.

[0207] In some examples, assume that data informative of the position of the marine object can be obtained using a sensor of the marine object or of the marine vessel, such as AIS, radar, or other adapted systems. In this case, the position of the marine object over time, as well as the position of the marine vessel over time, can be determined (the position of the marine vessel itself can be obtained using AIS or GPS). It is therefore possible to determine the CPA and / or the TCP A, which can be used in the label. This can include, for example, determining analytically the shortest distance between the marine vessel and the marine object.

[0208] Note that the determined CPA and / or TCPA can be also used to determine the collision risk in the label. For example, for a CPA over 0.6 Nm, the label can be selected as corresponding to a low collision risk, for a CPA between 0.3 Nm and 0.6 Nm, the label can be selected as corresponding to a medium collision risk, for a CPA equal to or less than 0.3 Nm, and a TCPA over 10 minutes, the label can be selected as corresponding to a medium collision risk, and for a CPA equal to or less than 0.3 Nm, and a TCPA above 10 minutes, the label can be selected as corresponding to a high collision risk. We may not that these values are not limitative in any way and can be adjusted as needed. In some examples, these values can be dynamically modified, according to the identification of the type of operations, or to other criteria. Similarly, classification of the collision risk into three levels is not limitative, and a different number of levels can be used. Note that although the training of the machine learning model may rely on data provided by non-imaging sensors (such as AIS), it has already been mentioned above that the trained machine learning model can determine the collision risk (and the CPA / TCPA) without using data informative of the marine object provided by non-imaging sensors.

[0209] The method further includes feeding (operation 830) to the first machine learning model 160, for each set of images Si: for each given marine obj ect MJ.SI in the given set Si of images, data Dbeanng informative of a direction of the given marine object MJ.SI in the given set of images Si and data Dimension informative of one or more dimensions of the marine object MJ.SI in the given set of images Si;

[0210] - the label associated with each given marine object MJ.SI in the given set Si of images.

[0211] Based on this input data (training data), the first machine learning model 160 tries to predict at least one of: (i) data Dnsk informative of a collision risk between the marine vessel and the marine object; (ii) a closest point of approach with the marine object; (iii) a time to the closest point of approach with the marine object.

[0212] A comparison of this prediction with the label can be used to update weights associated with neurons of the layers of the first machine learning model 160. Methods such as Backpropagation can be used to train the first machine learning model 160.

[0213] The weighting and / or threshold values of the first machine learning model 160 can be initially selected prior to training and can be further iteratively adjusted or modified during training to achieve an optimal set of weighting and / or threshold values in a trained neural network. After each iteration, a difference (also called loss function) can be determined between the actual output produced by the machine learning model 160 and the target output (values stored in the label) associated with the respective training set of data. The difference can be referred to as an error value. Training can be determined to be complete when a cost or loss function indicative of the error value is less than a predetermined value, or when a limited change in performance between iterations is achieved.

[0214] The first machine learning model 160 is therefore trained to output at least one of (i) a level of collision risk (for example, low, medium, high, or any other adapted classification), (ii) a closest point of approach with the marine object; (iii) a time to the closest point of approach with the marine object. In some examples, as explained with reference to Figs. 5, and 6A to 6D, a plurality of machine learning models MLk can be used, each trained with data generated over a period of time with a different duration Atk. The method of Fig. 9 can be used to train each machine learning model MLk, by feeding data Dbearing Atk and data Ddimension _Atk computed for a plurality of marine objects based on images collected during periods of time with this duration Atk.

[0215] In some examples, data Ddimension Atk is informative of a variation in the one or more dimensions of the marine object between the beginning and the end of the period of time Atk.

[0216] In other examples, data Ddimension Atk is informative of a variation in the one or more dimensions of the marine object in each image collected within the period of time Atk (in other words, for each image collected within the period of time Atk, the difference between the one or more dimensions of the marine object in this image and the previous image is determined).

[0217] In some examples, data Dbearing Atk is informative of a variation in the direction of the marine object between the beginning and the end of the period of time Atk.

[0218] In other examples, data Ddimension Atk is informative of a variation in the direction of the marine object in each image collected within the period of time Atk (in other words, for each image collected within the period of time Atk, the difference between the direction of the marine object in this image and the previous image is determined).

[0219] The label associated with each marine object within each period of time Atk includes data Dusk informative of a collision risk between the marine vessel and the marine object (provided e.g., by an operator), a closest point of approach with the marine object, and a time to the closest point of approach with the marine object. Note that the CPA and / or TCPA can have a single value in the label (corresponding to the estimate of CPA and / or TCPA in the last image of the sequence) or can be associated with different values (the estimate for CPA and / TCPA may evolve over time due to the dynamic motion of the marine vessel and / or of the marine object).

[0220] Note that each set of images Si can be used to train the different machine learning models MLk: the data Dbearing and Ddimension and the label are determined, for each machine learning model MLk, over a different period of time Atk within each set of image Si.

[0221] Note that this method can be used also to train the first machine learning model

[0222] 160. The first machine learning model 160 can be fed with data Dbearing Atk and data Ddimension Atk collected over periods of time with different durations Atk, together with labels associated with each marine object within each period of time Atk (data Dnsk, CPA and TCP A) for its training. The trained first machine learning model 160 is therefore able to receive data Dbeanng Atk and data Ddimension Atk collected over periods of time with different durations Atk, and to determine at least one of a level of collision risk (for example, low, medium, high, or any other adapted classification), (ii) a closest point of approach with the marine object; (iii) a time to the closest point of approach with the marine object. This is not limitative.

[0223] Attention is now drawn to Fig- 10, which describes a method of training the second machine learning model 162 (which can be used, during prediction, as explained with reference to Fig. 7).

[0224] The method includes obtaining (operation 1000) a training set. The training set includes one or more sets of images Si to SN. Each set of images Si includes a plurality of images of one or more marine objects MOi,sito MOp,si acquired by an imaging device of a marine vessel. Note that the different sets of images can be acquired by imaging devices mounted on different marine vessels. This is however not mandatory.

[0225] For each given set of images Si, for each given marine object MJ.SI in the given set Si of images, an area corresponding to the given marine object MJ.SI in each image of the given set is determined.

[0226] The method further includes feeding (operation 1010) to the second machine learning model 162, training data including:

[0227] - the one or more sets of images Si to SN; for each given set of images Si, for each given marine object MJ.SI in the given set Si of images, areas corresponding to the given marine obj ect MJ.SI in the images of the given set Si; for each set of images Si, and for each given marine object MJ.SI, a label indicative of at least one of: (i) a collision risk (e.g. high, medium, low, or any adapted label) between the marine vessel (on which the imaging device acquiring the set of images Si is located) and the given marine object MJ.SI; (ii) a closest point of approach with the given marine object; (iii) a time to the closest point of approach with the given marine object.

[0228] Based on this input data (training data), the second machine learning model 162 tries to predict at least one of: (i) data Dnsk informative of a collision risk between the marine vessel and each marine object; (ii) a closest point of approach with each marine object; (iii) a time to the closest point of approach with each marine object.

[0229] A comparison of this prediction with the label can be used to update weights associated with neurons of the layers of the second machine learning model 162. Methods such as Backpropagation can be used to train the second machine learning model 162.

[0230] The weighting and / or threshold values of the second machine learning model 162 can be initially selected prior to training and can be further iteratively adjusted or modified during training to achieve an optimal set of weighting and / or threshold values in a trained neural network. After each iteration, a difference (also called loss function) can be determined between the actual output produced by the second machine learning model 162 and the target output (values stored in the label) associated with the respective training set of data. The difference can be referred to as an error value. Training can be determined to be complete when a cost or loss function indicative of the error value is less than a predetermined value, or when a limited change in performance between iterations is achieved.

[0231] The second machine learning model 162 is therefore trained to output at least one of (i) a level of collision risk (for example, low, medium, high, or any other adapted classification) with a marine object, (ii) a closest point of approach with the marine object; (iii) a time to the closest point of approach with the marine object, based on images of the marine object and areas corresponding to the location of the marine object in these images.

[0232] In some examples, the second machine learning model 162 is also trained to determine the areas in which the marine objects are located throughout the images. This can be performed by introducing, in the labels fed to the second machine leaning model 162, the areas of the marine objects in the images.

[0233] As a consequence, it suffices to feed to the trained second machine learning model 162 a set of images of marine objects, and it outputs at least one of (i) a level of collision risk (for example, low, medium, high, or any other adapted classification) with each marine object, (ii) a closest point of approach with each marine object; (iii) a time to the closest point of approach with each marine object.

[0234] In the detailed description, numerous specific details are set forth in order to provide a thorough understanding of the invention. However, it will be understood by those skilled in the art that the presently disclosed subject matter may be practiced without these specific details. In other instances, well-known methods have not been described in detail so as not to obscure the presently disclosed subject matter.

[0235] Unless specifically stated otherwise, as apparent from the following discussions, it is appreciated that throughout the specification, discussions utilizing terms such as “obtaining”, “using”, “feeding”, “determining”, “estimating”, “training”, “transmitting”, “aggregating” or the like, refer to the action(s) and / or process(es) of at least one processing circuitry that manipulates and / or transforms data into other data, said data represented as physical, such as electronic, quantities and / or said data representing the physical objects.

[0236] The terms “computer” or “computer-based system” should be expansively construed to include any kind of hardware-based electronic device with a data processing circuitry (e.g., digital signal processor (DSP), a GPU, a TPU, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), microcontroller, microprocessor etc.), including, by way of non-limiting example, the computer-based system 101 of Fig. 1 and respective parts thereof disclosed in the present application. The data processing circuitry (designated also as processing circuitry - see e.g., reference 161 and / or 110) can comprise, for example, one or more processors operatively connected to computer memory, loaded with executable instructions for executing operations, as further described below. The data processing circuitry encompasses a single processor or multiple processors, which may be located in the same geographical zone, or may, at least partially, be located in different zones, and may be able to communicate together. The one or more processors can represent one or more general-purpose processing devices such as a microprocessor, a central processing unit, or the like. More particularly, a given processor may be one of a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, a processor implementing other instruction sets, or a processor implementing a combination of instruction sets. The one or more processors may also be one or more special-purpose processing devices such as an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), a network processor, or the like. The one or more processors are configured to execute instructions for performing the operations and steps discussed herein. The memories referred to herein can comprise one or more of the following: internal memory, such as, e.g., processor registers and cache, etc., main memory such as, e.g., read-only memory (ROM), flash memory, dynamic random access memory (DRAM) such as synchronous DRAM (SDRAM) or Rambus DRAM (RDRAM), etc.

[0237] The terms "memory" and “non-transitory computer readable medium” used herein should be expansively construed to cover any volatile or non-volatile computer memory suitable to the presently disclosed subject matter. The terms should be taken to include a single medium or multiple media (e.g., a centralized or distributed database, and / or associated caches and servers) that store the one or more sets of instructions. The terms shall also be taken to include any medium that is capable of storing or encoding a set of instructions for execution by the computer and that cause the computer to perform any one or more of the methodologies of the present disclosure. The terms shall accordingly be taken to include, but not be limited to, a read only memory (“ROM”), random access memory (“RAM”), magnetic disk storage media, optical storage media, flash memory devices, etc.

[0238] It is to be noted that while the present disclosure refers to the processing circuitry 161 (or 110) being configured to perform various functionalities and / or operations, the functionalities / operations can be performed by the one or more processors of the processing circuitry 161 (or 110) in various ways. By way of example, the operations described hereinafter can be performed by a specific processor, or by a combination of processors. The operations described hereinafter can thus be performed by respective processors (or processor combinations) in the processing circuitry 161 (or 110), while, optionally, at least some of these operations may be performed by the same processor. The present disclosure should not be limited to be construed as one single processor always performing all the operations.

[0239] When referring to operations performed by a processing circuitry (such as the processing circuitry 110), this can include a configuration in which the operations are performed locally (by a processing circuitry located on the marine vessel), or remotely (by one or more processors of a cloud, remote server, remote computerized system(s)), or partially locally and partially remotely.

[0240] In embodiments of the presently disclosed subject matter, fewer, more, and / or different stages than those shown in the methods of Figs. 2A, 3A, 5 and 7 to 10 may be executed. In embodiments of the presently disclosed subject matter, one or more stages illustrated in the methods of Figs. 2A, 3A, 5 and 7 to 10 may be executed in a different order, and / or one or more groups of stages may be executed simultaneously.

[0241] Embodiments of the presently disclosed subject matter are not described with reference to any particular programming language. It will be appreciated that a variety of programming languages may be used to implement the teachings of the presently disclosed subject matter as described herein.

[0242] The invention contemplates a computer program being readable by a computer for executing one or more methods of the invention. The invention further contemplates a machine-readable memory tangibly embodying a program of instructions executable by the machine for executing one or more methods of the invention.

[0243] It is to be noted that the various features described in the various embodiments may be combined according to all possible technical combinations.

[0244] It is to be understood that the invention is not limited in its application to the details set forth in the description contained herein or illustrated in the drawings. The invention is capable of other embodiments and of being practiced and carried out in various ways. Hence, it is to be understood that the phraseology and terminology employed herein are for the purpose of description and should not be regarded as limiting. As such, those skilled in the art will appreciate that the conception upon which this disclosure is based may readily be utilized as a basis for designing other structures, methods, and systems for carrying out the several purposes of the presently disclosed subject matter.

[0245] Those skilled in the art will readily appreciate that various modifications and changes can be applied to the embodiments of the invention as hereinbefore described without departing from its scope, defined in and by the appended claims.

Claims

CLAIMS1. A system comprising one or more processing circuitries configured to: obtain a plurality of images of at least one marine object, acquired by an imaging device of a marine vessel, use at least some of the plurality of images to determine: data Dbeanng informative of a direction of the at least one marine object, and data Ddimension informative of one or more dimensions of the at least one marine object in the at least some of the plurality of images, and use data Dbeanng and data Ddimension to determine at least one of:(i) data Dusk informative of a collision risk between the marine vessel and the marine object,(ii) data DCPA informative of a closest point of approach (CPA) between the marine vessel and the at least one marine object, or(iii) data DTCPA informative of a time to the closest point of approach (TCP A) between the marine vessel and the at least one marine object.

2. The system of claim 1, wherein data During is informative of a variation of the direction of the at least one marine object in the at least some of the plurality of images.

3. The system of claim 1 or of claim 2, wherein data Dbeanng is informative of a bearing angle variation of the marine object in the at least some of the plurality of images.

4. The system of any one of claims 1 to 3, wherein data Ddimension is informative of a variation in the one or more dimensions of the at least one marine object in the at least some of the plurality of images.

5. The system of any one of claims 1 to 4, wherein the one or more processing circuitries are operative to implement at least one machine learning model, wherein the one or more processing circuitries are configured to feed data Duringand data Dimension to the at least one machine learning model to determine, by the at least one machine learning model, at least one of data Dnsk, data DCPA or data DTCPA.

6. The system of claim 5, wherein the at least one machine learning model has been trained with a training set including: for each given marine object of a plurality of marine objects: data Dbearing informative of a direction of the given marine object in images of the given marine object acquired by an imaging device of a given marine vessel; data Ddimension informative of one or more dimensions of the given marine object in said images; at least one of: data Dnsk informative of a collision risk between the given marine vessel and the given marine object, data DCPA informative of a closest point of approach between the given marine vessel and the given marine object, or data DTCPA informative of a time to the closest point of approach between the given marine vessel and the given marine object.

7. The system of any one of claims 1 to 6, wherein the one or more processing circuitries are configured to: obtain a model, modelling a relationship between:(i) data Dbearing and data Ddimension of at least one given marine object, and(ii) at least one of data Dnsk informative of a collision risk between a given marine vessel and the at least one given marine object, or data DCPA informative of a closest point of approach between the given marine vessel and the at least one given marine object, or data DTCPA informative of a time to the closest point of approach between the given marine vessel and the given marine object;use the model, data Dbearing informative of a direction of the at least one marine object, and data Dimension informative of one or more dimensions of the at least one marine object, to determine at least one of data Dusk informative of a collision risk between the marine vessel and the at least one marine object, data DCPA informative of a closest point of approach between the marine vessel and the at least one marine object, or data DTCPA informative of a time to the closest point of approach between the marine vessel and the at least one marine object.

8. The system of any one of claims 1 to 7, configured to determine at least one of data Dnsk informative of a collision risk between the marine vessel and the at least one marine object, data DCPA informative of a closest point of approach between the marine vessel and the at least one marine object, or data DTCPA informative of a time to the closest point of approach between the marine vessel and the at least one marine object, without using data informative of the at least one marine object acquired by a sensor which is not a camera.

9. The system of any one of claims 1 to 8, configured to determine at least one of data Dnsk informative of a collision risk between the marine vessel and the at least one marine object, data DCPA informative of a closest point of approach between the marine vessel and the at least one marine object, or data DTCPA informative of a time to the closest point of approach between the marine vessel and the at least one marine object, without using data informative of the at least one marine object acquired by a radar system and an automatic identification system (AIS).

10. The system of any one of claims 1 to 9, configured to determine, for each given marine object of a plurality of marine objects present in the plurality of images, at least one of data Dnsk informative of a collision risk between the marine vessel and the given marine object, data DCPA informative of a closest point of approach between the marine vessel and the given marine object, or data DTCPA informative of a time to the closest point of approach between the marine vessel and the given marine object.

11. The system of any one of claims 1 to 10, configured to:use the at least some of the plurality of images to determine: data Dbearing short time and data Ddimension short time for the at least one marine object over a first period of time, and data Dbearing medium time and data Ddimension medium time for the at least one the marine object over a second period of time, wherein the second period of time is longer than the first period of time,USe data Dbearing short time, data Ddimension short time, data Dbearing medium time and data Ddimension medium time to determine at least one of:(i) data Dusk informative of a collision risk between the marine vessel and the at least one marine object,(ii) data DCPA informative of a closest point of approach between the marine vessel and the at least one marine object, or(iii) data DTCPA informative of a time to the closest point of approach between the marine vessel and the at least one marine object.

12. The system of claim 11, configured to feed data Dbearing shortjime, data Ddimension short time, data Dbearing medium time and data Ddimension _medium time to at least one trained machine learning model to determine, by the at least one trained machine learning model, at least one of:(i) data Drisk informative of a collision risk between the marine vessel and the at least one marine object,(ii) data DCPA informative of a closest point of approach between the marine vessel and the at least one marine object, or(iii) data DTCPA informative of a time to the closest point of approach between the marine vessel and the at least one marine object.

13. The system of any one of claims 1 to 12, configured to: use the at least some of the plurality of images to determine: data Dbearing short time and data Ddimension short time for the at least one marine object over a first period of time, and data Dbearing medium time and data Ddimension medium time for the at least one marine object over a second period of time, wherein the second period of time is longer than the first period of time, perform at least one of:(i) using data Dbeanng _shoit_time and data Dimension Jihort time to determine first data Dnsk i informative of a collision risk between the marine vessel and the at least one marine object and using data Dbearing medium time and data Ddimension medium time to determine second data Dnsk i informative of a collision risk between the marine vessel and the at least one marine object;(ii) using data Dbearing _short_time and data Ddimension _shoit time to determine first data DCPA i informative of a closest point of approach between the marine vessel and the at least one marine object and using data Dbearing medium time and data Ddimension medium time to determine second data DCPA 2 informative of a closest point of approach between the marine vessel and the at least one marine object; or(iii) using data Dbeanng shoit.time and data Ddimension -short-time to determine first data DTCPA 1 informative of a time to the closest point of approach between the marine vessel and the at least one marine object and using data Dbearing medium time and data Ddimension medium time to determine second data DTCPA 2 informative of a time to the closest point of approach between the marine vessel and the at least one marine object.

14. The system of claim 13, configured to perform at least one of(i) aggregating the first data Dnsk 1 informative of a collision risk between the marine vessel and the at least one marine object and the second data Dnsk 2 informative of a collision risk between the marine vessel and the at least one marine object to determine data Dnsk informative of a collision risk between the marine vessel and the at least one marine object;(ii) aggregating the first data DCPA 1 informative of a closest point of approach between the marine vessel and the at least one marine object and the second data DCPA 2 informative of a closest point of approach between the marine vessel and the at least one marine object to determine data DCPA informative of a closest point of approach between the marine vessel and the at least one marine object;(iii) using data Dbeanng short tae and data Ddimension -short-time to determine first data DTCPA 1 informative of a time to the closest point of approach between the marine vessel and the at least one marine object and using data Dbeanng medium time and data Ddimension medium time to determine second data DTCPA 2 informative of a time to theclosest point of approach between the marine vessel and the at least one marine object.

15. The system of any one of claims 1 to 14, wherein the one or more processing circuitries are operative to implement a first machine learning model and a second machine learning model different from the first machine learning model, wherein the system is configured to: use the at least some of the plurality of images to determine: data Dbearing short time and data Ddimension short time for the at least one marine object over a first period of time, and data Dbearing medium time and data Ddimension medium time for the at least one marine object over a second period of time, wherein the second period of time is longer than the first period of time, perform at least one of:(i) feeding data Dbearing short time and data Ddimension -Short time tO the first machine learning model to determine first data Dnsk i informative of a collision risk between the marine vessel and the at least one marine object and feeding data Dbearing medium time and data Ddimension medium time tO the second machine learning model to determine second data Dnsk 2 informative of a collision risk between the marine vessel and the at least one marine object;(ii) feeding data Dbearing short time and data Ddimension short time tO the first machine learning model to determine first data DCPA 1 informative of a closest point of approach between the marine vessel and the at least one marine object and feeding data Dbearing medium time and data Ddimension medium time to the second machine learning model to determine second data DCPA 2 informative of a closest point of approach between the marine vessel and the at least one marine object; or(ifi) feeding data Dbearing short time and data Ddimension short time tO the first machine learning model to determine first data DTCPA 1 informative of a time to the closest point of approach between the marine vessel and the at least one marine object and feeding data Dbearing medium time and data Ddimension medium time to the second machine learning model to determinesecond data DTCPA 2 informative of a time to the closest point of approach between the marine vessel and the at least one marine object.

16. The system of claim 15, wherein at least one of (i) or (ii) is met:(i) the first machine learning model has been trained using data Dbearing and data Ddimension determined for one or more marine objects based on images collected over periods of time matching the first period of time according to a criterion, and(ii) the second machine learning model has been trained using data Dbearing and data Ddimension determined for one or more marine objects based on images collected over periods of time matching the second period of time according to a criterion.

17. The system of any one of claims 1 to 16, configured to determine at least one of (i) data Drisk informative of a collision risk between the marine vessel and the at least one marine object, (ii) data DCPA informative of a closest point of approach (CPA) between the marine vessel and the at least one marine object, or (iii) data DTCPA informative of a time to the closest point of approach (TCP A) between the marine vessel and the at least one marine object, without estimating a distance between the marine vessel and the at least one marine object.

18. The system of any one of claims 1 to 17, configured to transmit at least one of (i) data Dnsk informative of a collision risk between the marine vessel and the at least one marine object, (ii) data DCPA informative of a closest point of approach (CPA) between the marine vessel and the at least one marine object, or (iii) data DTCPA informative of a time to the closest point of approach (TCP A) between the marine vessel and the at least one marine object to a computer-implemented controller operative to control at least one of a position, a direction, a velocity, or an acceleration of the marine vessel.

19. The system of claim 18, wherein the controller is operative to use at least one of data Drisk, data DCPA or data DTCPA to modify at least one of at least one of a position, a direction, a velocity or an acceleration of the marine vessel, to avoid a collision or a near-collision with the at least one marine object.

20. The system of any one of claims 1 to 19, configured to display: one or more images of the plurality of images of the at least one marine object, acquired by the imaging device of the marine vessel, informative data superimposed on said on or more images, the informative data including at least one of: data Dnsk informative of a collision risk between the marine vessel and the at least one marine object, data DCPA informative of a closest point of approach (CPA) between the marine vessel and the at least one marine object, or data DTCPA informative of a time to the closest point of approach (TCP A) between the marine vessel and the at least one marine object.

21. The system of any one of claims 1 to 20, configured to repetitively recalculate data Dbearing informative of a direction of the at least one marine object, and data Dimension informative of one or more dimensions of the at least one marine object over time, and to update an estimate of (i) data Dnsk informative of a collision risk between the marine vessel and the at least one marine object, (ii) data DCPA informative of a closest point of approach (CPA) between the marine vessel and the at least one marine object, or (iii) data DTCPA informative of a time to the closest point of approach between the marine vessel and the at least one marine object.

22. A marine vessel comprising: an imaging device, and one or more processing circuitries configured to: obtain a plurality of images of at least one marine object, acquired by the imaging device of the marine vessel, use at least some of the plurality of images to determine: data Dbearing informative of a direction of the at least one marine object, and data Dtfimension informative of one or more dimensions of the marine obj ect in the at least some of the plurality of images, and use data Dbearing and data Dimension to determine at least one of:(i) data Dnsk informative of a collision risk between the marine vessel and the at least one marine object,(ii) data DCPA informative of a closest point of approach (CPA) between the marine vessel and the at least one marine object, or(iii) data DTCPA informative of a time to the closest point of approach (TCP A) between the marine vessel and the at least one marine object.

23. A system comprising one or more processing circuitries configured to implement at least one machine learning model, wherein the one or more processing circuitries are configured to: obtain a plurality of images of at least one marine object, acquired by an imaging device of a marine vessel, feed at least some of the plurality of images to the machine learning model to determine at least one of:(i) data Dnsk informative of a collision risk between the marine vessel and the at least one marine object,(ii) data DCPA informative of a closest point of approach (CPA) between the marine vessel and the at least one marine object, or(iii) data DTCPA informative of a time to the closest point of approach (TCP A) between the marine vessel and the at least one marine object.

24. The system of claim 23, configured to, for each image of the at least some of the plurality of images: determine an area in which the at least one marine object is located, and feed the area to the machine learning model.

25. The system of claim 23 or of claim 24, wherein the machine learning model has been trained using a training set including, for each given marine object of a plurality of marine objects: images of the given marine object acquired by an imaging device of a given marine vessel, at least one of:(i) data Dusk informative of a collision risk between the given marine vessel and the given marine object;(ii) data DCPA informative of a closest point of approach (CPA) between the given marine vessel and the given marine object, or(iii) data DTCPA informative of a time to the closest point of approach (TCP A) between the given marine vessel and the given marine object.

26. A controller of a marine vessel, the controller comprising one or more processing circuitries configured to: obtain at least one of:(i) data Dusk informative of a collision risk between the marine vessel and at least one marine object, determined based on data Dbearing informative of a direction of the at least one marine object in images acquired by an imaging device of the marine vessel, and data Dimension informative of one or more dimensions of the at least one marine object in said images;(ii) data DCPA informative of a closest point of approach (CPA) between the marine vessel and the at least one marine object, determined based on data Dbeanng informative of a direction of the at least one marine object in images acquired by an imaging device of the marine vessel, and data Dimension informative of one or more dimensions of the at least one marine object in said images; or(iii) data DTCPA informative of a time to the closest point of approach (TCP A) between the marine vessel and the at least one marine object, determined based on data Dbearing informative of a direction of the at least one marine object in images acquired by an imaging device of the marine vessel, and data Dimension informative of one or more dimensions of the at least one marine object in said images; use at least one of the data Dusk, the data DCPA or the data DTCPA to control at least one of a position, a velocity, or an acceleration of the marine vessel to avoid a collision with the at least one marine object.

27. A method comprising, by one or more processing circuitries:obtaining a plurality of images of at least one marine object, acquired by an imaging device of a marine vessel, using at least some of the plurality of images to determine: data Dbeanng informative of a direction of the at least one marine object, and data Ddimension informative of one or more dimensions of the at least one marine object in the at least some of the plurality of images, and using data Dbeanng and data Ddimension to determine at least one of:(i) data Dusk informative of a collision risk between the marine vessel and the at least one marine object,(ii) data DCPA informative of a closest point of approach (CPA) between the marine vessel and the at least one marine object, or(iii) data DTCPA informative of a time to the closest point of approach (TCP A) between the marine vessel and the at least one marine object.

28. The method of claim 27, wherein data During is informative of a variation of the direction of the at least one marine object in the at least some of the plurality of images.

29. The method of claim 27 or of claim 28, wherein data Dbearing is informative of a bearing angle variation of the at least one marine object in the at least some of the plurality of images.

30. The method of any one of claims 27 to 29, wherein data Ddimension is informative of a variation in the one or more dimensions of the at least one marine object in the at least some of the plurality of images.

31. The method of any one of claims 27 to 30, comprising feeding data Dbearing and data Ddimension to at least one machine learning model to determine, by the at least one machine learning model, at least one of data Dnsk, data DCPA or data DTCPA.

32. The method of claim 31, wherein the at least one machine learning model has been trained with a training set including:for each given marine object of a plurality of marine objects: data Dbearing informative of a direction of the given marine object in images of the given marine object acquired by an imaging device of a given marine vessel; data Ddimension informative of one or more dimensions of the given marine object in said images; at least one of: data Dnsk informative of a collision risk between the given marine vessel and the given marine object, data DCPA informative of a closest point of approach between the given marine vessel and the given marine object, or data DTCPA informative of a time to the closest point of approach between the given marine vessel and the given marine object.

33. The method of any one of claims 27 to 32, comprising: obtaining a model modelling a relationship between:(i) data Dbearing and data Ddimension of at least one given marine object, and(ii) at least one of data Dnsk informative of a collision risk between a given marine vessel and the at least one given marine object, or data DCPA informative of a closest point of approach between the given marine vessel and the at least one given marine object, or data DTCPA informative of a time to the closest point of approach between the given marine vessel and the at least one given marine object; using the model, data Dbearing informative of a direction of the at least one marine object, and data Ddimension informative of one or more dimensions of the at least one marine object to determine at least one of: data Dnsk informative of a collision risk between the marine vessel and the at least one marine object, data DCPA informative of a closest point of approach between the marine vessel and the at least one marine object, or data DTCPAinformative of a time to the closest point of approach between the marine vessel and the at least one marine object.

34. The method of any one of claims 27 to 33, comprising determining at least one of data Drisk informative of a collision risk between the marine vessel and the at least one marine object, data DCPA informative of a closest point of approach between the marine vessel and the at least one marine object, or data DTCPA informative of a time to the closest point of approach between the marine vessel and the at least one marine object, without using data informative of the at least one marine object acquired by a sensor which is not a camera.

35. The method of any one of claims 27 to 34, comprising determining at least one of data Dnsk informative of a collision risk between the marine vessel and the at least one marine object, data DCPA informative of a closest point of approach between the marine vessel and the at least one marine object, or data DTCPA informative of a time to the closest point of approach between the marine vessel and the at least one marine object, without using data informative of the marine object acquired by a radar system and an automatic identification system (AIS).

36. The method of any one of claims 27 to 35, comprising determining, for each given marine object of a plurality of marine objects present in the plurality of images, at least one of data Drisk informative of a collision risk between the marine vessel and the given marine object, data DCPA informative of a closest point of approach between the marine vessel and the given marine object, or data DTCPA informative of a time to the closest point of approach between the marine vessel and the given marine object.

37. The method of any one of claims 27 to 36, comprising: using the at least some of the plurality of images to determine: data Dbearing short time and data Dimension short time for the at least one marine object over a first period of time, and data Dbearing medium time and data Dimension medium time for the at least one marine object over a second period of time, wherein the second period of time is longer than the first period of time,Using data Dbearing short time, data Ddimension short time, data Dbearing medium time and data Ddimension medium time to determine at least one of:(i) data Dnsk informative of a collision risk between the marine vessel and the at least one marine object,(ii) data DCPA informative of a closest point of approach between the marine vessel and the at least one marine object, or(iii) data DTCPA informative of a time to the closest point of approach between the marine vessel and the at least one marine object.

38. The method of claim 37, comprising feeding data Dbeanng shonjime, data Ddimension short time, data Dbeanng medium time and data Dimension medium time to a trained machine learning model to determine, by the trained machine learning model, at least one of:(i) data Dnsk informative of a collision risk between the marine vessel and the at least one marine object,(ii) data DCPA informative of a closest point of approach between the marine vessel and the at least one marine object, or(iii) data DTCPA informative of a time to the closest point of approach between the marine vessel and the at least one marine object.

39. The method of any one of claims 27 to 38, comprising: using the at least some of the plurality of images to determine: data Dbeanng short time and data Ddimension short time for the at least one marine object over a first period of time, and data Dbeanng medium time and data Ddimension medium time for the at least one marine object over a second period of time, wherein the second period of time is longer than the first period of time, performing at least one of:(i) using data Dbeanng shoitjime and data Dimension _short_time to determine first data Dnsk i informative of a collision risk between the marine vessel and the at least one marine object and using data Dbearing medium time and data Ddimension medium time to determine second data Dnsk i informative of a collision risk between the marine vessel and the at least one marine object;(ii) using data Dbearing_short_time and data Ddimension _short time to determine first data DCPA i informative of a closest point of approach between the marine vessel and the at least one marine object and using data Dboaring medium timo and data Ddimension medium time to determine second data DCPA 2 informative of a closest point of approach between the marine vessel and the at least one marine object; or(iii) using data Dbeanng _shoit_time and data Ddimension _siioit time to determine first data DTCPA 1 informative of a time to the closest point of approach between the marine vessel and the at least one marine object and using data Dbearing medium time and data Ddimension medium time to determine second data DTCPA 2 informative of a time to the closest point of approach between the marine vessel and the at least one marine object.

40. The method of claim 39, comprising performing at least one of(i) aggregating the first data Dnsk 1 informative of a collision risk between the marine vessel and the at least one marine object, and the second data Dnsk 2 informative of a collision risk between the marine vessel and the at least one marine object, to determine data Dnsk informative of a collision risk between the marine vessel and the at least one marine object;(ii) aggregating the first data DCPA 1 informative of a closest point of approach between the marine vessel and the at least one marine object, and the second data DCPA 2 informative of a closest point of approach between the marine vessel and the at least one marine object, to determine data DCPA informative of a closest point of approach between the marine vessel and the at least one marine object;(iii) using data Dbeanng short tae and data Ddimension -short-time to determine first data DTCPA 1 informative of a time to the closest point of approach between the marine vessel and the at least one marine object, and using data Dbeanng medium time and data Ddimension medium time to determine second data DTCPA 2 informative of a time to the closest point of approach between the marine vessel and the at least one marine object.

41. The method of any one of claims 27 to 40, comprising: using the at least some of the plurality of images to determine:data Dbearing short time and data Ddimension short time for the at least one marine object over a first period of time, and data Dbearing medium time and data Ddimension medium time for the at least one marine object over a second period of time, wherein the second period of time is longer than the first period of time, performing at least one of(i) feeding data Dbearing _shoit_time and data Ddimension _shoit_time to a first machine learning model to determine first data Dnsk i informative of a collision risk between the marine vessel and the at least one marine object and feeding data Dbearing medium time and data Ddimension medium time to a second machine learning model to determine second data Dnsk 2 informative of a collision risk between the marine vessel and the at least one marine object;(ii) feeding data Dbearing short time and data Ddimension short time tO the first machine learning model to determine first data DCPA 1 informative of a closest point of approach between the marine vessel and the at least one marine object and feeding data Dbearing medium time and data Ddimension medium time to the second machine learning model to determine second data DCPA 2 informative of a closest point of approach between the marine vessel and the at least one marine object; or(ifi) feeding data Dbearing short time and data Ddimension short time tO the first machine learning model to determine first data DTCPA 1 informative of a time to the closest point of approach between the marine vessel and the at least one marine object, and feeding data Dbearing medium time and data Ddimension medium time to the second machine learning model to determine second data DTCPA 2 informative of a time to the closest point of approach between the marine vessel and the at least one marine object.

42. The method of any one of claims 27 to 41, comprising determining at least one of (i) data Dnsk informative of a collision risk between the marine vessel and the at least one marine object, (ii) data DCPA informative of a closest point of approach (CPA) between the marine vessel and the at least one marine object, or (iii) data DTCPA informative of a time to the closest point of approach (TCP A) between the marine vessel and the at least one marine object, without estimating a distance between the marine vessel and the at least one marine object.

43. The method of any one of claims 27 to 42, comprising transmitting at least one of (i) data Drisk informative of a collision risk between the marine vessel and the at least one marine object, (ii) data DCPA informative of a closest point of approach (CPA) between the marine vessel and the at least one marine object, or (iii) data DTCPA informative of a time to the closest point of approach (TCP A) between the marine vessel and the at least one marine object to a computer-implemented controller operative to control at least one of a position, a direction, a velocity, or an acceleration of the marine vessel.

44. The method of any one of claims 27 to 43, comprising displaying: one or more images of the plurality of images of the at least one marine object, acquired by the imaging device of the marine vessel, informative data superimposed on said one or more images, the informative data including at least one of: data Dnsk informative of a collision risk between the marine vessel and the at least one marine object, data DCPA informative of a closest point of approach (CPA) between the marine vessel and the at least one marine object, or data DTCPA informative of a time to the closest point of approach (TCP A) between the marine vessel and the at least one marine object.

45. The method of any one of claims 27 to 44, comprising repetitively recalculating data Dbearing informative of a direction of the at least one marine object, and data Ddimension informative of one or more dimensions of the at least one marine object over time, and updating an estimate of (i) data Drisk informative of a collision risk between the marine vessel and the at least one marine object, (ii) data DCPA informative of a closest point of approach (CPA) between the marine vessel and the at least one marine object, or (iii) data DTCPA informative of a time to the closest point of approach between the marine vessel and the at least one marine object.

46. A method comprising, by one or more processing circuitries:obtaining a plurality of images of at least one marine object, acquired by an imaging device of a marine vessel, feeding at least some of the plurality of images to a machine learning model to determine at least one of:(i) data Dusk informative of a collision risk between the marine vessel and the at least one marine object,(ii) data DCPA informative of a closest point of approach (CPA) between the marine vessel and the at least one marine object, or(iii) data DTCPA informative of a time to the closest point of approach (TCP A) between the marine vessel and the at least one marine object.

47. The method of claim 46, comprising, for each image of the at least some of the plurality of images: determining an area in which the marine object is located, and feeding the area to the machine learning model.

48. The method of claim 46 or of claim 47, wherein the machine learning model has been trained using a training set including, for each given marine object of a plurality of marine objects: images of the given marine object acquired by an imaging device of a given marine vessel, at least one of:(i) data Dusk informative of a collision risk between the given marine vessel and the given marine object;(ii) data DCPA informative of a closest point of approach (CPA) between the given marine vessel and the given marine object, or(iii) data DTCPA informative of a time to the closest point of approach (TCP A) between the given marine vessel and the given marine object.

49. A non-transitory computer readable medium comprising instructions that, when executed by one or more processing circuitries, cause the one or more processing circuitries to perform:obtaining a plurality of images of at least one marine object, acquired by an imaging device onboard a marine vessel, using at least some of the plurality of images to determine: data Dbeanng informative of a direction of the at least one marine object, and data Ddimension informative of one or more dimensions of the at least one marine object in the at least some of the plurality of images, and using data Dbeanng and data Ddimension to determine at least one of:(i) data Dusk informative of a collision risk between the marine vessel and the at least one marine object,(ii) data DCPA informative of a closest point of approach (CPA) between the marine vessel and the at least one marine object, or(iii) data DTCPA informative of a time to the closest point of approach (TCP A) between the marine vessel and the at least one marine object.

50. A non-transitory computer readable medium comprising instructions that, when executed by one or more processing circuitries, cause the one or more processing circuitries to perform: obtaining a plurality of images of at least one marine object, acquired by an imaging device onboard a marine vessel, feeding at least some of the plurality of images to the machine learning model to determine at least one of:(i) data Dusk informative of a collision risk between the marine vessel and the at least one marine object,(ii) data DCPA informative of a closest point of approach (CPA) between the marine vessel and the at least one marine object, or(iii) data DTCPA informative of a time to the closest point of approach (TCP A) between the marine vessel and the at least one marine object.

Images