Use of high-resolution images to enable AI identification of ships

Abstract

An object detection method comprises: (A) receiving image(s) captured by imaging device(s) mounted on a water platform, wherein the image(s) include object(s) located within a body of water; (B) (i) a low-resolution detection process, the low-resolution detection process comprises (1) receiving reduced-resolution image(s) generated by performing resolution reduction; (2) performing object detection using a neural network to obtain detected object(s). (ii) a high-resolution detection process based on dividing image(s) into image tiles, the high-resolution detection process comprises (1) receiving a set of tiles derived from image(s) - each tile corresponds to a subsection of image(s), and the spatial resolution of each tile is higher than the reduced spatial resolution of the reduced-resolution image(s); (2) performing tile-based object detection on the tiles using an image tile neural network(s) to obtain tile-based detected object(s). (iii) performing a process including merging the object(s) detected at reduced resolution with the tile-based object(s) to obtain a final set of detected objects.

Description

Technology Field

[0001] The subject matter of the patent disclosed herein relates to the field of marine navigation. Background Technology

[0002] Some of the major concerns in the field of navigation include preventing a surface vessel or other platform from colliding with other surface vessels, platforms, or other obstacles and objects, particularly in the event of human observer inattention.

[0003] The acknowledgment of the above references in this application should not be inferred to mean that these references have any relationship with the patentability of the subject matter disclosed herein.

[0004] The following are exemplary embodiments of the subject matter of the patent disclosed herein.

[0005] According to a first embodiment of the subject matter of the patent disclosed herein, a computerized object detection method is provided, wherein the method is performed by a processing circuit of an object detection system, and

[0006] A) Receiving one or more images captured by at least one imaging device mounted on a water platform -

[0007] One or more images include one or more objects located within a body of water -;

[0008] B)

[0009] i. Performing a low-resolution detection process - The low-resolution detection process is,

[0010] 1. Receiving one or more reduced-resolution images generated by performing image resolution reduction, which includes reducing the spatial resolution of one or more images; and

[0011] 2. Includes performing object detection on one or more reduced-resolution images using a neural network to acquire one or more detected objects detected at the reduced resolution.

[0012] ii. Performing a high-resolution detection process based on dividing one or more images into image tiles - the high-resolution detection process is,

[0013] 1. Receiving a set of image tiles derived from at least one of one or more images -

[0014] Each image tile in the set corresponds to a subsection of at least one image, and

[0015] The spatial resolution of each image tile is higher than the reduced spatial resolution of one or more reduced resolution images -; and

[0016] 2. Performing tile-based object detection on a set of image tiles using at least one image tile neural network to acquire one or more tile-based detected objects -; and

[0017] iii. A step of performing a process including merging one or more detection objects detected at reduced resolution with one or more tile-based detection objects to obtain a final set of detection objects.

[0018] In addition to the features above, a method according to such an embodiment of the subject matter of the patent disclosed herein may include one or more of the features (i) through (lxiv) listed below in any technically possible desired combination or permutation.

[0019] (i) One or more images include one or more high-resolution images, and at least one imaging device includes at least one high-resolution imaging device.

[0020] (ii) The method is,

[0021] C) Includes a step of deriving an image tile set.

[0022] (iii) The method is,

[0023] D) Includes a step of performing image resolution reduction.

[0024] (iv) The method is,

[0025] E) For one or more local inquiry and segmentation (LIAS) definition regions associated with an image tile set, performing LIAS at a resolution higher than the reduced resolution using at least one local inquiry neural network to acquire one or more local inquiry-based detection objects -

[0026] One or more LIAS defined regions represent the detection of one or more tile-based detection objects in more than one image tile, and

[0027] Each LIAS defined area includes a portion of at least two image tiles -; and

[0028] F) By merging a set of image tile-based detection objects with one or more local lookup-based detection objects,

[0029] It further includes steps to facilitate the acquisition of an updated final detection object set.

[0030] (v) Local lookup and segmentation includes generating at least one additional image tile corresponding to one or more defined regions, and performing local lookup and segmentation within at least one additional image tile.

[0031] (vi) One or more LIAS defined regions represent the detection of one or more tile-based detection objects in the overlapping regions of image tiles.

[0032] (vii) Image(s) are captured in real time, and the system is configured to perform the method in one of real time; near real time; and online,

[0033] It facilitates collision avoidance between a water platform and one of one or more objects.

[0034] (viii) The system is configured to perform the method within 0.01 seconds to 1 second.

[0035] (ix) The system is configured to perform the method within 1 to 60 seconds.

[0036] (x) The method is repeated at least once more.

[0037] (xi) The system is configured to synchronize input from at least one imaging device with input from at least one of an inertial measurement unit (IMU) and an inertial navigation system (INS).

[0038] (xii) The method is,

[0039] G) Includes a step of performing tracking of one or more objects based on iteration.

[0040] (xiii) A water platform is a water vessel.

[0041] (xiv) A watercraft is one of the ships and boats.

[0042] (xv) One or more objects include at least one of other water platforms, other water vessels, buoys, obstacles and lighthouses.

[0043] (xvi) The final set of detected objects includes at least one of other surface platforms, other surface vessels, buoys, obstacles, and lighthouses.

[0044] (xvii) At least one imaging device is configured to observe one or more objects at an observation resolution associated with a human observer.

[0045] (xviii) One or more captured images have a resolution of at least 640 pixels in at least one dimension.

[0046] (xix) One or more images have a horizontal resolution of at least 640 pixels.

[0047] (xx) One or more images have a vertical resolution of at least 640 pixels.

[0048] (xxi) One or more captured images have a resolution of at least 1080 pixels in at least one dimension.

[0049] (xxii) One or more images have a horizontal resolution of at least 1080 pixels.

[0050] (xxiii) One or more images have a vertical resolution of at least 1080 pixels.

[0051] (xxiv) One or more images have a horizontal resolution of at least 1280 pixels.

[0052] (xxv) One or more images have a vertical resolution of at least 1280 pixels.

[0053] (xxvi) One or more images have a horizontal resolution of at least 4000 pixels.

[0054] (xxvii) One or more images have a vertical resolution of at least 4000 pixels.

[0055] (xxviii) One or more images have a horizontal resolution of at least 6000 pixels.

[0056] (xxix) One or more images have a vertical resolution of at least 6000 pixels.

[0057] (xxx) Image resolution reduction includes reducing at least one of the horizontal resolution and vertical resolution of one or more captured images.

[0058] (xxxi) At least some adjacent image tiles of an image tile set have at least partial horizontal overlap and / or at least partial vertical overlap.

[0059] (xxxii) At least some adjacent image tiles of an image tile set have at least partial horizontal overlap.

[0060] (xxxiii) At least some adjacent image tiles in the set have at least partial vertical overlap.

[0061] (xxxiv) At least some adjacent image tiles of an image tile set are in contact with each other.

[0062] (xxxv)

[0063] [a] At least some adjacent image tiles of an image tile set have at least partial horizontal overlap and / or at least partial vertical overlap; and

[0064] (xxxvi) [b] At least one of the following is true: at least some adjacent image tiles in an image tile set are in contact with each other. At least some adjacent image tiles in an image tile set do not have horizontal overlap and vertical overlap.

[0065] (xxxvii) Each image tile contains fewer horizontal pixels than the number of horizontal pixels of at least one image.

[0066] (xxxviii) Each image tile contains fewer vertical pixels than the number of vertical pixels of at least one image.

[0067] (xxxix) An image tile set corresponds to every pixel of one or more captured images.

[0068] (xl) The low-resolution detection process is performed before the high-resolution detection process.

[0069] (xli) The low-resolution detection process is performed in parallel with the high-resolution detection process.

[0070] (xlii) The low-resolution detection process is performed after the high-resolution detection process.

[0071] (xliii) The low-resolution detection process is performed prior to the LIAS-based detection process.

[0072] (xliv) The low-resolution detection process is performed in parallel with the LIAS-based detection process.

[0073] (xlv) The low-resolution detection process is performed after the LIAS-based detection process.

[0074] (xlvi) neural networks and at least one image tile neural network have the same size.

[0075] (xlvii) The neural network and at least one image tile neural network are identical.

[0076] (xlviii) At least one image tile neural network and at least one local lookup neural network have the same size.

[0077] (xlix) At least one image tile neural network and at least one local lookup neural network are identical.

[0078] (l) The neural network and at least one local lookup neural network have the same size.

[0079] (li) The neural network and at least one local lookup neural network are identical.

[0080] (lii) The method is,

[0081] H) further includes a step of classifying at least one detection object of the final set.

[0082] (liii) The method is,

[0083] I) further includes the step of identifying at least one detection object of the final set.

[0084] (liv) At least one imaging device is configured to cover an angle of up to 360 degrees around the water platform.

[0085] (lv) At least one imaging device is configured to cover at least 180 degrees around the water platform.

[0086] (lvi) At least one imaging device is configured to cover at least 270 degrees around the water platform.

[0087] (lvii) At least one imaging device is configured to cover 360 degrees around the water platform.

[0088] (lviii) At least one imaging device is included in at least one payload, each payload includes one or more imaging devices, and at least one payload is configured to cover up to a 360-degree angle around the water platform.

[0089] (lix) At least one imaging device is included in a plurality of payloads, and each payload includes a plurality of imaging devices.

[0090] (lx) At least one imaging device is a camera.

[0091] (lxi) The object detection system includes at least one imaging device.

[0092] (lxii) The method is,

[0093] J) further includes the step of merging the final detection object set with an additional final detection object set associated with image capture at different field of view (FOV) of at least one imaging device.

[0094] (lxiii) The method is,

[0095] K) further includes a step of outputting information representing the final set of detected objects.

[0096] (lxiv) The output information includes location information associated with at least one detection object of the final set.

[0097] According to a second aspect of the subject matter of the patent disclosed herein, a computerized object detection method is provided, wherein the method is performed by a processing circuit of an object detection system, and

[0098] A) Receiving one or more captured images captured by at least one imaging device mounted on a water platform -

[0099] One or more images include one or more objects located within a body of water -;

[0100] B)

[0101] i. Step of performing the low-resolution detection process - The low-resolution detection process is,

[0102] 1. Receiving one or more reduced-resolution images generated by performing image resolution reduction, which includes reducing the spatial resolution of one or more images; and

[0103] 2. Includes performing object detection on one or more reduced-resolution images using a neural network to obtain zero or more detected objects detected at the reduced spatial resolution.

[0104] ii. A step of performing a high-resolution detection process by dividing one or more images into image tiles - the high-resolution detection process is,

[0105] 1. Receiving a set of image tiles derived from at least one of one or more images -

[0106] Each image tile in the set corresponds to a subsection of at least one image, and

[0107] The image tile spatial resolution of each image tile is higher than the reduced spatial resolution of one or more reduced resolution images -; and

[0108] 2. Includes performing tile-based object detection on a set of image tiles using at least one image tile neural network to acquire one or more tile-based detected objects.

[0109] iii. A step of performing a process including merging zero or more detection objects detected at reduced spatial resolution with one or more tile-based detection objects to obtain a final set of detection objects.

[0110] According to a third embodiment of the subject matter of the patent disclosed herein, a computerized object detection method is provided, wherein the method is performed by a processing circuit of an object detection system, and

[0111] A) Receiving one or more images captured by at least one imaging device mounted on a water platform -

[0112] One or more images include one or more objects located within a body of water -;

[0113] B)

[0114] i) Performing a high-resolution detection process based on dividing one or more images into image tiles - the high-resolution detection process is,

[0115] (1) Receiving a set of image tiles derived from at least one of one or more images -

[0116] Each image tile in the set corresponds to a subsection of at least one image, and

[0117] The spatial resolution of each image tile is higher than the reduced spatial resolution of one or more reduced resolution images -; and

[0118] (2) Performing tile-based object detection on a set of image tiles using at least one image tile neural network to obtain one or more tile-based detection objects -;

[0119] ii) For one or more Local Query and Segmentation (LIAS) definition regions associated with an image tile set, performing LIAS at a resolution higher than the reduced resolution using at least one Local Query neural network to acquire one or more Local Query-based detection objects -

[0120] One or more LIAS defined regions represent the detection of one or more tile-based detection objects in more than one image tile, and

[0121] Each defined area includes a portion of at least two image tiles -; and

[0122] iii) By merging a set of image tile-based detection objects with one or more local lookup-based detection objects,

[0123] It includes a step of performing a process that includes obtaining a final set of detected objects.

[0124] According to a fourth embodiment of the subject matter of the patent disclosed herein, a computerized object detection method is provided, wherein the method is performed by a processing circuit of an object detection system, and

[0125] A) Receiving one or more images captured by at least one imaging device mounted on a water platform -

[0126] One or more images include one or more objects located within a body of water -;

[0127] B)

[0128] i. Performing a low-resolution detection process - The low-resolution detection process is,

[0129] 1. Receiving one or more reduced-resolution images generated by performing image resolution reduction, which includes reducing the spatial resolution of one or more images; and

[0130] 2. Includes performing object detection on one or more reduced-resolution images using a neural network to acquire one or more detected objects detected at the reduced resolution.

[0131] ii. Performing a high-resolution detection process based on dividing one or more images into image tiles - the high-resolution detection process is,

[0132] (1) Receiving a set of image tiles derived from at least one of one or more images -

[0133] Each image tile in the set corresponds to a subsection of at least one image, and

[0134] The spatial resolution of each image tile is higher than the reduced spatial resolution of one or more reduced resolution images -; and

[0135] (2) Performing tile-based object detection on a set of image tiles using at least one image tile neural network to obtain one or more tile-based detection objects -;

[0136] iii. For one or more Local Query and Segmentation (LIAS) definition regions associated with an image tile set, performing LIAS at a resolution higher than the reduced resolution using at least one Local Query neural network to acquire one or more Local Query-based detection objects -

[0137] One or more LIAS defined regions represent the detection of one or more tile-based detection objects in more than one image tile, and

[0138] Each LIAS defined area includes a portion of at least two image tiles -; and

[0139] iv. By merging one or more detection objects detected at reduced resolution with a set of image tile-based detection objects and one or more local lookup-based detection objects,

[0140] It includes a step of performing a process that includes obtaining a final set of detected objects.

[0141] In addition to the features above, the method according to the second to fourth embodiments of the subject matter of the patent disclosed herein may include one or more of the features (i) through (lxiv) listed above in any technically possible desired combination or permutation.

[0142] According to the fifth embodiment of the subject matter of the patent disclosed herein, a computerized object detection system is provided that includes a processing circuit configured to perform the method of any one of the first to fourth embodiments of the subject matter of the patent disclosed herein.

[0143] According to the sixth aspect of the subject matter of the patent disclosed herein, a non-transient computer-readable storage medium is provided that embodies in a tangible form a program of instructions that, when executed by a processing circuit of an object detection system, cause the processing circuit to perform a method of any one of the first to fourth aspects of the subject matter of the patent disclosed herein.

[0144] The fifth and sixth embodiments of the subject matter of the patent disclosed herein may optionally include one or more of the features (i) through (lxiv) listed above in any technically possible desired combination or permutation. Brief explanation of the drawing

[0145] In order to understand the present invention and to know how the present invention can be practically carried out, embodiments will be described through non-limiting examples with reference to the accompanying drawings. Fig. 1 This schematically illustrates an exemplary generalized view of ship navigation according to some embodiments of the subject matter of the patent disclosed herein. Fig. 2 This schematically illustrates an exemplary generalized schematic diagram of a water platform according to some embodiments of the subject matter of the patent disclosed herein. Fig. 3a and Fig. 3bThis schematically illustrates an exemplary generalized schematic diagram of a computerized object detection system according to some embodiments of the subject matter of the patent disclosed herein. Fig. 4 This schematically illustrates an exemplary generalized representation of an image according to some embodiments of the subject matter of the patent disclosed herein. Fig. 5 This schematically illustrates an exemplary generalized representation of a reduced-resolution image according to some embodiments of the subject matter of the patent disclosed herein. Figures 6a and 6b This schematically illustrates an exemplary generalized representation of an image tile according to some embodiments of the subject matter of the patent disclosed herein. Fig. 7 [This] schematically illustrates an exemplary generalized representation of a data flow according to some embodiments of the subject matter of the patent disclosed herein. FIGS. 8a to 8c This schematically illustrates a generalized flowchart of a process or method for object detection according to some embodiments of the subject matter of the patent disclosed herein. Specific details for implementing the invention

[0146] In the presented drawings and description, the same reference numerals indicate components common to different embodiments or configurations.

[0147] In the following detailed description, various specific details are provided to provide a thorough understanding of the invention. However, those skilled in the art will understand that the subject matter disclosed herein may be practiced without such specific details. In other cases, well-known methods, procedures, components, and circuits are not described in detail so as not to obscure the subject matter disclosed herein.

[0148] It should be understood that the present invention is not limited to the details presented in the description included herein or illustrated in the drawings in its application. Other embodiments of the invention are possible and may be practiced and carried out in various ways. Accordingly, it should be understood that the language and terminology used herein are for illustrative purposes only and should not be construed as limiting. Accordingly, those skilled in the art will recognize that the concepts on which this disclosure is based can be readily utilized as a basis for designing other structures, methods, and systems to achieve various purposes of the subject matter disclosed herein.

[0149] Furthermore, it will be understood that the system according to the present invention may be implemented at least partially on a suitably programmed computer. Likewise, the present invention also considers a computer program readable by a computer for carrying out the method of the present invention. The present invention further considers a non-transient computer-readable memory that embodies a program of instructions executable by a computer for carrying out the method of the present invention in a tangible form.

[0150] Those skilled in the art will readily recognize that various modifications and changes may be applied to the embodiments of the present invention as described above, without departing from the scope of the appended claims and the scope defined by those claims.

[0151] Unless otherwise specifically stated, as is evident from the discussion below, discussions throughout the specification utilizing terms such as “providing,” “receiving,” “performing,” “deriving,” “merging,” “selecting,” “outputting,” “classifying,” “identifying,” “computing,” “recomputing,” “calculating,” “determining,” “generating,” etc. are recognized as referring to the operation(s) and / or process(s) of a computer that manipulate and / or convert data representing a physical object into other data, for example, expressed in physical quantities such as electronic or mechanical quantities. The term “computer” refers to personal computers, servers, computing systems, communication devices, processors or processing units (e.g., digital signal processors (DSPs), microcontrollers, microprocessors, field programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), etc.), and computerized systems or devices disclosed in this application ( 300 ) and for example 320 It should be broadly interpreted to encompass any kind of hardware-based electronic device having data processing capabilities, including any other electronic computing device that includes a processing circuit section such as as a non-limiting example.

[0152] Operations according to the teachings of the present invention may be performed by a computer specifically configured for a desired purpose, or by a general-purpose computer specifically configured for a desired purpose by a computer program stored on a non-transient computer-readable storage medium.

[0153] The embodiments of the subject matter disclosed herein are not described with reference to any specific programming language. As described herein, it will be recognized that various programming languages ​​may be used to implement the teachings of the subject matter disclosed herein.

[0154] The terms “non-transient memory” and “non-transient storage medium” as used herein should be interpreted broadly to encompass any volatile or non-volatile computer memory suitable for the subject matter of the patent disclosed herein.

[0155] As used herein, the phrases “for example,” “same as,” “for example,” and variations thereof describe non-limiting embodiments of the subject matter disclosed herein. References to “one case,” “some case,” “other case,” “one example,” “some example,” “other example,” or variations thereof in this specification mean that a particular method, procedure, component, structure, feature, or characteristic described in relation to the embodiment(s) is included in at least one embodiment of the subject matter disclosed herein, but is not necessarily included in all embodiments. The appearance of the same term does not necessarily refer to the same embodiment(s) or example(s).

[0156] The use of conditional expressions, such as “may,” “may,” or variations thereof, should be interpreted as conveying that while one or more examples of the patentable subject matter may include a particular method, procedure, component, and feature, one or more other examples of the patentable subject matter may not necessarily include them. Accordingly, such conditional expressions are generally not intended to imply that the described particular method, procedure, component, or circuit is necessarily included in all examples of the patentable subject matter. Furthermore, the use of non-conditional expressions is not intended to imply that the described particular method, procedure, component, or circuit is necessarily included in all examples of the patentable subject matter.

[0157] It is recognized that for clarity, specific embodiments, methods, procedures, components, or features of the subject matter disclosed herein, described in the context of separate embodiments or examples, may also be provided in combination in a single embodiment or example. Conversely, for brevity, various embodiments, methods, procedures, components, or features of the subject matter disclosed herein, described in the context of a single embodiment, may also be provided separately or in any suitable sub-combination.

[0158] Furthermore, it should be noted that each of the drawings and the textual descriptions thereof are merely non-limiting examples for the sake of clarity of explanation, and describe one aspect of the subject matter disclosed herein only for informational purposes. It will be understood that the teachings of the subject matter disclosed herein are not limited to those described by reference to any of the drawings or those described in other documents referenced in this application.

[0159] With this in mind, Fig. 1 With reference to [this], FIG. 1 schematically illustrates an exemplary generalized view of ship navigation according to some embodiments of the subject matter of the patent disclosed herein. View ( 100 ) illustrates a conceptual and schematic exemplary view of ship navigation. In some examples, a surface platform ( 110 ) is one of the watercraft, for example, ships and boats. A ship, boat, or other watercraft ( 110 ) is a specific direction( 108 Moves / navigates in waters (not explicitly shown) as ). Ship ( 110 ) turns to the left (port side) 117 ...is doing. Water navigation body ( 110 ) is also referred to herein as a surface vessel. In some examples, a surface operating body ( 110 ) is a marine operating body ( 110 It is. In many parts of the disclosure of this application, the award platform ( 110) will be exemplified by specific examples of watercraft. In other non-limiting examples, water platforms ( 110 ) is a drilling platform, for example, offshore oil drilling equipment or other offshore-based platforms.

[0160] One or more human observers on a watercraft ( 115 ), for example, a vessel monitoring other platforms in the water (e.g., drilling platforms), other vessels, boats, obstacles and other objects ( 110 An observer or other crew member is on board. Several exemplary such objects are depicted for illustrative purposes only. In some examples, the observed object(s) include at least one of other platforms, e.g., other surface vessels, buoys, obstacles, and lighthouses.

[0161] Watercraft ( 110 A buoy relatively far from ) 125 ) and water navigation body( 110 Floating obstacle located to the right of ) 127 ) is depicted. Relatively large vessels ( 120 ) and small boats( 123 ) is also depicted, and both of these are watercraft ( 110 It is located approximately in front of the ). Watercraft ( 110 On the rear left side of ) boat( 140 ) is also cityd, and this boat( 140 ) is a water navigation vehicle ( 110 Turn to the left of ) 144 ...is doing. Water navigation body ( 110 To the left of ) is a relatively small boat ( 130 There is ), and this small boat( 130 ) is a large object further away ( 133 ), for example, visually obscures part of an aircraft carrier, ocean-going passenger ship, or large oil tanker.

[0162] obstacle( 127Examples of ) include floating cargo containers that have fallen from container ships, floating trees, and icebergs. In some examples, the detected object is a lighthouse protruding into the water.

[0163] In some examples, an automated computerized object detection system that detects images captured from an imaging device such as a camera ( 300 Deploying ) may have at least certain technical advantages. Such a system allows skilled human observers ( 115 ), for example, the monitor(s)( 115 It can replace ) or, in other cases, provide support to human monitors. For example, an object detection system ( 300 ) can detect surrounding objects even when a human observer is inattentive. As another example, an object detection system ( 300 ) can detect a relatively large number of objects in real time, even in cases where humans may struggle to detect all objects in a timely manner. In another example, this system can replace humans to reduce labor costs or cover shift work where human monitors cannot be utilized. Such detection is for ships ( 110 It can play an auxiliary role in navigation and / or facilitate collision avoidance.

[0164] In some non-limiting examples, imaging devices ( 210 It is configured to capture objects with a size of 3 to 5 meters located at a distance of 3 kilometers (km). Such requirements may be a factor in determining the resolution of the imaging device used.

[0165] In some examples, Fig. 2As further disclosed herein with reference to, the imaging device(s) are configured to capture a wide field of view that is not merely limited to the forward of the vessel, for example, such that the device can see to the sides and, in some cases, even to the rear, and in some examples, is configured to capture a 360-degree field of view. This may provide, for example, exemplary advantages in collision avoidance. As one example, the vessel ( 110 ) makes a sharp turn to the left ( 117 In the case of ), the vessel ( 110 ) is another vessel ( 133 If ) is not visible, other vessels ( 133 It can collide with ). As another example, the current vessel ( 110 The boat at the rear of ) 140 ) is a ship( 110 Turning toward the left side of ) 144 In the case of ), boat( 140 ) is a ship( 110 ) rotates ( 117 As a result of ) the vessel ( 110 It may cause collisions with ). Also, it should be noted that objects / ships are not depicted according to scale.

[0166] At least to solve such technical problems and provide such advantages, the present invention provides a computerized object detection method as well as a computerized object detection system for performing such a method ( 300 ) and software products are disclosed. In some examples, the method is,

[0167] A) One or more images ( 400 Step of receiving ) - In some examples, the image has high resolution, for example, having a spatial resolution of at least 640 pixels in at least one dimension, and such captured image(s)( 400 ) is one or more objects located within the water body ( 645 , 657 Includes ), and image(s) are award platforms( 110At least one imaging device mounted on ) 210 Captured by ) -;

[0168] B) Includes a step of performing the following processes.

[0169] i) Performing a low-resolution detection process. This first process is,

[0170] 1. One or more reduced-resolution images generated by performing image resolution reduction ( 500 Receiving ) - image resolution reduction includes reducing the spatial resolution of the captured image(s) (e.g., M×N) to a reduced resolution (e.g., m×n) -; and

[0171] 2. Neural Network ( 327 Reduced resolution image using ) 500 Perform object detection on ) and one or more detected objects ( 557 Includes obtaining ). These objects ( 557 ) is detected at reduced resolution.

[0172] ii) Captured image(s)( 400 ) image tile( 601 Performing a high-resolution detection process by dividing into ). This second process is,

[0173] 1. At least one image ( 400 Image tiles derived from ) 601 ) set( 600 Receiving ) - Each image tile in the set is a captured image ( 400 It corresponds to a subsection of ), and the image tile space resolution (e.g., M×N) of the image tile(s) is reduced resolution image(s)( 500 Reduced spatial resolution of ) (e.g., m×n) 601 Higher than ) -; and

[0174] 2. At least one image tile neural network ( 327Using ) set( 600 ) image tile( 607 , 601 Perform tile-based object detection on ) and one or more tile-based detected objects ( 680 , 670 Includes obtaining ).

[0175] iii) One or more detected objects detected at reduced spatial resolution ( 557 ) and one or more tile-based detection objects( 680 Performing a process that includes merging ) to obtain a final set of detected objects.

[0176] Note that in some examples, no objects are detected in either of the two processes.

[0177] In some examples, the method is an image tile set ( 600 It further includes a step of deriving ) and / or performing image resolution reduction.

[0178] In some examples, the method (C) provides information representing the final set of detected objects, e.g., at least one detected object of the final set ( 680 It further includes a step of outputting location information associated with ).

[0179] In some examples, the method is,

[0180] D) Image Tile Set ( 600 One or more definition domains associated with ) 612 For ), at least one local lookup neural network ( 327 It further includes a step of performing local lookup and segmentation (LIAS) at a spatial resolution higher than the reduced resolution (m×n) using ).

[0181] This definition domain is also referred to herein as the LIAS definition domain.

[0182] The definition area(s) is more than one image tile ( 601 ,602 Tile-based detection object(s) in ) 670 Represents the detection of ). Each defined region ( 618 ) is at least 2 image tiles ( 604 , 605 Includes a portion of ). In some embodiments, the definition area(s) are image tiles ( 604 , 602 , 607 , 608 The overlapping area of ​​) 663 Tile-based detection object(s) in ) 690 Indicates the detection of ). In some embodiments, the defined region(s) are image tiles ( 605' , 606' Tile-based detection object(s) in the contact area of ​​) 690 Indicates the detection of ). By doing so, this step obtains an updated final set of detected objects in some embodiments. The LIAS method is also referred to herein as a local search. In some embodiments, the local search is the LIAS defined region(s)( 612 At least one additional image tile corresponding to ) 614 It includes generating ) and performing local search within at least one additional image tile.

[0183] In addition, the present invention includes other computerized object detection methods as well as a computerized object detection system for performing such methods ( 300 ) and software products are disclosed. These other methods are, in some examples:

[0184] A) Award Platform( 110 ) receiving one or more images captured by at least one imaging device mounted on ) - one or more images include one or more objects located within the body of water -;

[0185] B)

[0186] iv) Performing a high-resolution detection process based on dividing one or more images into image tiles - the high-resolution detection process is,

[0187] (1) Receiving a set of image tiles derived from at least one of one or more images -

[0188] Each image tile in the set corresponds to a subsection of at least one image, and

[0189] The spatial resolution of each image tile is higher than the reduced spatial resolution of one or more reduced resolution images -; and

[0190] (2) Performing tile-based object detection on a set of image tiles using at least one image tile neural network to obtain tile-based detected object(s);

[0191] v) For one or more Local Lookup and Segmentation (LIAS) definition regions associated with an image tile set, performing LIAS at a resolution higher than the reduced resolution using at least one Local Lookup neural network to acquire one or more Local Lookup-based detection objects -

[0192] One or more LIAS defined regions represent the detection of one or more tile-based detection objects in more than one image tile, and

[0193] Each LIAS defined area includes a portion of at least two image tiles -; and

[0194] vi) includes a step of performing a process including merging an image tile-based detection object set with one or more local lookup-based detection objects to obtain a final detection object set.

[0195] In some implementations, the method is repeated one or more additional times. This is because the method is based on the repetition of object(s)( 120 , 123It can facilitate an implementation example that includes additional steps for performing tracing of ).

[0196] Various drawings illustrate the concept of the method disclosed herein. Figures 2 and 3 The present invention provides a schematic diagram of a system configured to perform the method of the patent subject to the invention. FIGS. 4 to 6 ... discloses exemplary examples of various types of images, and Fig. 7 It discloses an exemplary data flow associated with the method. FIGS. 8a to 8c This discloses an exemplary flowchart of a cross-validation method using a core set tree.

[0197] Additional advantages of the subject matter of the patent disclosed herein are further disclosed herein by reference to one or more of the drawings mentioned above.

[0198] now Fig. 2 With reference to FIG. 2, a water platform (according to some embodiment of the subject matter of the patent disclosed herein) 110 ), for example, an exemplary generalized schematic diagram of a watercraft ( 200 ) schematically illustrates. In some examples, the watercraft has one or more inertial measurement units (IMUs) ( 242 Includes ) and one or more inertial measurement units (IMUs) 242 ) is included in one or more inertial navigation systems (INS) (240) in some embodiments. In some examples, a surface platform ( 110 ) is a navigation system( 243 Includes ). In some examples, water platforms ( 110 ) is a collision avoidance system ( 245 Includes ). In other examples not illustrated in the drawings, a collision avoidance system ( 245 ) is a navigation system( 243 It is included within ). Examples of such systems are known in the art.

[0199] In some examples, water platforms ( 110 ) is an object detection system(s)(300 Includes ). System( 300 For more detailed information on the examples of ), Fig. 3a and Fig. 3b Further disclosed herein with reference to

[0200] In some examples, the platform ( 110 ) is one or more imaging devices ( 210 , 224 ), including, for example, cameras. In some examples, multiple imaging devices exist. For convenience of explanation, the drawings show reference numerals only for some of the devices illustrated in the drawings. In a non-limiting example of the drawings, the surface vessel includes 12 such imaging devices and 4 payloads ( 220 Each of the 3 imaging devices ( 210 Includes ).

[0201] Accordingly, in some embodiments, the system includes multiple payloads, each payload includes multiple imaging devices, and each of the multiple imaging devices has its own field of view. Such multiple payloads are combined so that up to 360 degrees of observation can be achieved in the combined state. In some embodiments, the camera or other imaging device is located in the same plane, for example, in the same horizontal plane.

[0202] In some embodiments, the system includes a plurality of payloads, each of which includes one or more imaging devices.

[0203] In some embodiments, the system includes a single payload, and the payload includes more than one imaging device.

[0204] Accordingly, at least one imaging device is included in at least one payload, and each payload includes one or more imaging devices.

[0205] In some embodiments, the imaging device(s) are configured to collectively cover an angle of up to 360 degrees around the water platform across all devices. Accordingly, in some embodiments, one or more payloads are configured to collectively cover an angle of up to 360 degrees around the water platform across all payloads and devices. In a specific example of the drawing, a set of imaging devices is configured to capture an image covering 360 degrees around the water platform. For example, each of the 12 devices has an angle of view of, for example, 38 degrees, and there is overlap between their angles of view.

[0206] In another example, the surface platform is composed of fewer cameras, e.g., fewer payloads, and the imaging device(s) view a smaller overall field of view. Thus, in some exemplary embodiments, the set of imaging devices is a surface vessel ( 110 It is configured to cover at least 180 degrees around ). For example, a surface vehicle with two additional cameras ( 226 and 228 3 device payloads facing forward together with ) 222 In cases including ), the watercraft navigation direction ( 108 Objects within a field of view exceeding 180 degrees centered on ) can be detected. In another exemplary embodiment, the imaging device set is configured to cover at least 270 degrees around the surface vessel. Other possible configurations are possible.

[0207] In some examples, the imaging device field of view (FOV) of two adjacent imaging devices A , B Some or all of them overlap each other as illustrated. In some other examples, adjacent viewing angles B , C Some or all of them do not overlap each other. Therefore, line( 280 and 284) shows the boundaries of viewing angles that do not overlap or touch each other, and a gap exists in the viewing angle. Therefore, in the drawing, the line ( 280 ) and line( 284 It can be seen that there is a gap between ).

[0208] In another example, some or all of the adjacent viewing angles do not overlap but rather touch. That is, the adjacent boundaries of the viewing angles ( 280 , 284 ) come into contact with each other, and therefore the line( 280 and 284 ) is identical. Such an embodiment is not shown in the drawings for the sake of brevity of description.

[0209] In addition, instead of using multiple cameras to cover a wide field of view, the watercraft uses one or more fixed cameras ( 220 Instead of ) operating imaging device(s)( 230 Note that, for example, it can be configured to have one or more gimbal cameras. The use of such movable cameras can facilitate an increase in the overall system field of view compared to the field of view of a single individual imaging device. Gimbal imaging device ( 230 ) indicated by a dashed line indicates that the device is an alternative option to the embodiment shown by a solid line in the drawing. In other non-limiting examples, a pan-tilt camera ( 230 ) or pan-tilt-roll camera( 230 ) is utilized.

[0210] Fig. 1 As disclosed with reference to, the water platform ( 110 Imaging a significant portion or most of the peripheral field of view of ) is, for example, a ship ( 110 It should be noted that it can provide at least a certain advantage when trying to avoid a collision with a boat on the side or rear of ).

[0211] In some examples, the watercraft is a communication network ( 247Includes ) and communication network( 247 ) is an award platform( 110 Various components of ), for example, the system ( 300 , 240 , 242 , 243 , 245 , 247 ) and various imaging devices( 210 , 226 , 227 , 224 , 228 It is configured to communicate between ).

[0212] In some examples, the detection system ( 300 ) is imaging device(s)( 210 Input from ) to IMU( 242 ) and / or INS( 240 It is configured to synchronize with the input of ). Such synchronization can match a specific image captured at a specific time t with IMU / INS data associated with that specific time t (or, for example, the time closest to t). This is, for example, a surface vessel at capture time t ( 110 It can be advantageous when determining the orientation of ).

[0213] For the sake of brevity of explanation, the subject matter of the patent disclosed herein is a water vessel ( 110 It is described with reference to ). In another embodiment, the surface vessel is a submarine. In another exemplary embodiment, the imaging device ( 210 Instead, it is installed on an aerial vehicle, such as an airplane, quadcopter, drone, or balloon. In another exemplary embodiment, the imaging device is installed instead on land, for example, on a coast, that is, in a place with an optical view of the water. For example, the imaging device is mounted on a lighthouse or tower.

[0214] now Fig. 3a and Fig. 3bWith reference to FIGS. 3a and FIGS. 3b, a computerized object detection system according to some embodiments of the subject matter of the patent disclosed herein ( 300 This schematically illustrates an exemplary generalized diagram of ). In some non-limiting examples, a computerized system ( 300 ) includes computers. Computerized systems ( 300 ) is a non-limiting example of a processing circuit section ( 320 This processing circuit includes the processor ( 330 ) and memory( 325 It may include ).

[0215] This processing circuit section ( 320 ) may, in non-limiting examples, be general-purpose computer(s) specifically configured for a desired purpose by a computer program stored on a non-transient computer-readable storage medium. They may be configured to execute various function modules according to computer-readable instructions. In other non-limiting examples, this processing circuit ( 320 ) can be computer(s) specially configured for a desired purpose.

[0216] In some embodiments, the processing circuit ( 320 ) is one or more graphics processing units (GPUs) 323 It also includes ). In some examples, the GPU includes one or more neural networks (NN)( 327 Includes ). Various types of NNs, e.g., Deep Neural Network (DNN)(s) or other networks known as themselves, may be utilized. In some examples, three different neural networks, or three different groups / sets of neural networks ( 327 ) exist, and each of them is configured for a specific function. For example, separate neural network(s) may exist for each of the following steps.

[0217] I. A step of performing object detection at reduced spatial resolution on one or more reduced-resolution images (500) as part of a low-resolution detection process. In some examples, these neural network(s) are also referred to herein as reduced-resolution or low-resolution neural network(s). Further details regarding this Fig. 5 , Fig. 7 and Fig. 8 It is disclosed by referring to .

[0218] II. A step of performing tile-based object detection on image tiles to obtain one or more tile-based detected objects. In some examples, this neural network(s) is referred to herein as image tile neural network(s). Further details regarding this Fig. 6 , Fig. 7 and Fig. 8 It is disclosed by referring to .

[0219] III. A step of performing Local Lookup and Segmentation (LIAS) at a spatial resolution higher than the reduced resolution. In some cases, this yields one or more Local Lookup-based detection objects, e.g., a set of Local Lookup-based detection objects. In some cases, this yields an updated final set of detection objects. In some examples, this neural network(s) is referred to herein as Local Lookup Neural Network(s). Further details regarding this Fig. 6 , Fig. 7 and Fig. 8 It is disclosed by referring to .

[0220] In some exemplary embodiments, the reduced resolution neural network(s) and the image tile neural network(s) have the same size. In some embodiments, the reduced resolution neural network(s) and the local lookup neural network(s) have the same size. In some embodiments, the image tile neural network(s) and the local lookup neural network(s) have the same size. In some embodiments, the reduced resolution neural network(s) and the image tile neural network(s) are the same neural network(s). In some embodiments, the reduced resolution neural network(s) and the local lookup neural network(s) are the same neural network(s). In some embodiments, the image tile neural network(s) and the local lookup neural network(s) are the same neural network(s). Note that in some cases, determining the size of the NN and the method of sharing it are engineering decisions to optimize the time and system resources involved.

[0221] Therefore, for example, using the same neural network for both reduced-resolution detection and image tile detection can reduce computer resource usage, but it may take more time because certain tasks must be performed sequentially rather than in parallel. Additionally, it should be noted that the number of neural networks utilized within a specific function of the process is an engineering decision. Therefore, for example, Fig. 6 In the case where 15 image tiles are used, 15 NNs enable more complete parallel processing in image tile-based detection, but will require more resources compared to running a single neural network 15 times. Similarly, more than 1 and fewer than 15 NNs may be utilized.

[0222] In other examples, other machine learning techniques, such as decision trees or random forests, are used instead of neural networks to perform detection.

[0223] processor(s) 330 ) and / or GPU(s)( 323 )Is Fig. 3bVarious functional modules can be executed as additionally disclosed herein by referring to [the document].

[0224] In some examples, the processing circuit ( 320 ) memory( 325 ) is configured to store data associated with the object detection process, for example, relatively transient data. Non-limiting examples of the data stored are watercraft ( 110 It includes the orientation of ), the estimated size and distance of the object, and pixels associated with each detected object. The memory also includes, for example, a gimbal camera for each image capture ( 230 The observation angle and synchronized capture time t of ) can be stored.

[0225] In some examples, object detection systems ( 300 ) is a data store( 390 Includes ). In some examples, longer-term and continuous data is stored in a data repository. An illustrated, non-limiting example is data tracked over time, each fixed camera ( 210 )'s viewing direction and angle, viewing angle, and detected object( 120 Includes classification and / or identification information associated with ).

[0226] now Fig. 3b Referring to, the processing circuit section ( 320 ) processor(s)( 330 ) and / or GPU(s)( 323 Possible modules of ) are illustrated. Processor(s)( 330 ) and / or GPU(s)( 323 ) can execute various functional modules. All modules are processor(s)( 330 Can be executed on ), or all modules are GPU(s)( 323 Can be executed in ) or in any suitable combination of processor(s)( 330 ) and GPU(s)( 323It is also possible to split modules between ). This choice is an engineering decision in some cases. In some examples, functional modules are data repositories ( 390 It is a software module stored in ), and at execution, the function module is in memory ( 325 Loaded into ) processor(s)( 330 ) and / or GPU(s)( 323 It is executed by ).

[0227] In some examples, components ( 330 , 320 ) is an input / output (I / O) module( 380 Includes ). In some examples, the module ( 380 ) is an imaging device( 210 Input from ), and possibly IMU / INS( 240 , 242 It is configured to receive input from ). Exemplary outputs are detection objects and various alarms (e.g., regarding detected problem situations), which are, for example, navigation systems ( 243 ) and / or collision system( 245 It is transmitted to ). In other embodiments, a separate input / output module (not shown) exists. In some embodiments, these modules are in the system ( 300 It can communicate with an I / O interface (not shown) included in ), and the I / O interface may be, for example, various possible types of physical interfaces known in the art. Communication is, for example, communication network(s)( 247 It can be performed through ).

[0228] In some examples, components ( 330 , 320 ) is the captured image interface and control module ( 333 Includes ). This module includes metadata such as capture time, for example, and the I / O module( 380 Receive images captured via ) and / or imaging device ( 210It is configured to transmit a command to control ), and this command is, for example, an imaging device ( 210 Indicates the point in time to capture an image to ) or gimbal device ( 230 This includes instructing the device to rotate to a different viewing angle. In other embodiments, there is a separate module (not shown) for receiving the captured image and controlling the imaging device.

[0229] In some examples, components ( 330 , 320 ) is a resolution reduction module( 335 This module includes ). This module is, for example, Fig. 5 Captured image(s) as disclosed with reference to 400 It is configured to reduce the spatial resolution of ).

[0230] In some examples, components ( 330 , 320 ) is a reduced resolution detection module ( 337 This module includes ). This module is, for example, Fig. 5 As disclosed with reference to , the neural network ( 327 Reduced resolution image(s) using ) 500 It is configured to perform object detection on ).

[0231] In some examples, components ( 330 , 320 ) is a tile forming module( 340 This module includes ). This module is, for example, Fig. 6 Captured image(s) as disclosed with reference to 400 It is configured to derive image tiles from ).

[0232] In some examples, components ( 330 , 320 ) is a tile-based detection module ( 345 This module includes ). This module is, for example, Fig. 6 As disclosed with reference to, for example, image tile neural network(s)( 327It is configured to detect objects within image tiles using ).

[0233] In some examples, components ( 330 , 320 ) is a local search module ( 348 Includes ) and local search module ( 348 ) is the Local Inquiry and Segmentation (LIAS) module at this institution ( 348 It is also referred to as ). This module is, for example, Fig. 6 As disclosed with reference to, for example, local query neural network(s)( 327 It is configured to perform a local lookup on image tiles, for example, when an object is detected in more than one image tile using ).

[0234] In some examples, components ( 330 , 320 ) is the result merging module( 345 This module includes ). This module is, for example, Fig. 7 and Fig. 8 As disclosed by reference, the method is configured to merge objects detected in, for example, in reduced-resolution image(s), in image tiles, and / or in local lookups / segmentations. In other exemplary embodiments, the method involves several different mergings, each of which uses a different merging module.

[0235] In some examples, components ( 330 , 320 ) is the iteration and tracking module( 355 Includes ). This module is, for example, a module ( 335 , 337 , 340 , 345 , 348 , 350 It is configured to control the repetition of the detection process performed by ) etc. In some examples, the module is also, for example, Fig. 8 Track the movement of an object within a body of water as disclosed by referring to [the source].

[0236] In some examples, components ( 330 , 320 ) is a classification and identification module ( 358 This module includes ). This module is, for example, Fig. 8 It is configured to perform classification and / or identification of detection objects as disclosed with reference to [the source]. In some other implementations, separate modules exist for classification and identification, respectively.

[0237] Fig. 3a and Fig. 3b ...is illustrated only with a general schematic diagram of the system architecture, which is a non-limiting example solely for the sake of clarity of explanation and describes specific aspects of the subject matter disclosed herein in an informative manner. The teachings of the subject matter disclosed herein are Fig. 3a and Fig. 3b It will be understood that you are not bound by the content described by referring to [this].

[0238] Only specific components necessary to exemplify the subject matter of the patent disclosed herein are illustrated. Other components and sub-components not illustrated may exist. Fig. 3a and Fig. 3b A system such as the system described in connection with non-limiting examples may perform all, part, or part of the methods disclosed herein.

[0239] Fig. 3a and Fig. 3b Each system component and module may be composed of any combination of software, hardware, and / or firmware as applicable, executed on a suitable device or on devices that performs the functions defined and described herein. The hardware may be digital and / or analog. As described for each system component and module, equivalent and / or modified functions may be integrated or divided in other ways. Accordingly, in some embodiments of the subject matter disclosed herein, the system Fig. 3a and Fig. 3bCompared to that shown in, it may include fewer, more, modified, and / or different components, modules, and functions. To provide a non-limiting example of this, in some examples, a resolution reduction module ( 335 ) and reduced resolution detection module( 337 ) is combined into a single module. Similarly, in some cases, the module ( 380 ) can instead be two modules, namely an input module and an output module. Similarly, in some cases, the module ( 355 ) is separated into two modules, namely an iteration module and a tracking module.

[0240] One or more of these components and modules may, as applicable, be centralized in a single location or distributed across more than one location. In some examples, specific components utilize cloud implementations and are implemented, for instance, in a private or public cloud.

[0241] Fig. 3a and Fig. 3b Each component may represent multiple specific components of a possible distributed architecture configured to operate independently and / or cooperatively to process various data and electrical inputs and enable operations related to computerized object detection. In some cases, multiple instances of a component may be utilized for reasons of performance, redundancy, and / or availability. Similarly, in some cases, multiple instances of a component may be utilized for reasons of function or application. For example, different parts of a specific function may be deployed in different instances of a component.

[0242] Fig. 3a and Fig. 3bWhere the various components of the system are not located collectively in a single location or on a single physical component, communication between the various components may be realized by any signaling system or communication component, module, protocol, software language, and driving signal, and may suitably be wired communication and / or wireless communication. This is an I / O module ( 380 Interfaces and systems such as ) 300 The same applies to the external interface (not shown) of ).

[0243] now Fig. 4 Note that FIG. 4 is an image according to some embodiment of the subject matter of the patent disclosed herein ( 400 An exemplary generalized expression of ) 400 ) schematically exemplifies. The image is an imaging device ( 210 It was captured by ), and therefore the image captured at this institution ( 400 It is referred to as ). The captured image has a spatial resolution of M×N pixels (horizontal × vertical). An exemplary image contains multiple objects. Object( 660 and 665 ) are relatively small in the image because, for example, they are small and / or distant objects, such as buoys or fishing boats located far away. Objects( 645 ) is of relatively medium size. Object( 657 and 630 ) is a relatively large object, for example, an ocean liner. A smaller boat or other object ( 655 ) is an object( 657 Note that it is in front. Object( 653 ) is an object ( on the captured image 650 Partially blocks or hides ). object( 640 and 643 ) appears to be in contact with each other on the image.

[0244] In some examples, one or more images ( 400) is one or more high-resolution images ( 400 Includes ) and at least one imaging device ( 210 ) is at least one high-resolution imaging device ( 210 Includes ). Non-limiting examples of "high resolution" are disclosed herein. In some examples, an image ( 400 ) has a spatial resolution of at least 640 pixels in at least one dimension, e.g., in the horizontal and / or vertical direction. This level of resolution is considered a “high” resolution for the purposes of the subject matter disclosed herein. In some examples, one or more of the images have a resolution of at least 1080 pixels in at least one dimension. In some examples, one or more of the images have a resolution of at least 4000 pixels in at least one dimension. In some examples, one or more of the images have a resolution of at least 6000 pixels in at least one dimension. In some examples, one or more of the images have a resolution of at least 8000 pixels in at least one dimension. Examples include 6000×3376 pixels and 4000×2250 pixels. These are non-limiting examples.

[0245] It can be technically advantageous to detect images using the highest available device resolution. If cost is a consideration, it can be technically advantageous to detect images using the highest resolution available on standard cameras in the desired price range. As camera or other device technology advances, cameras at various price points ( 210 Note that the resolution achievable with ) increases. Detection of high-resolution images is object( 660 and 665 It has at least an exemplary technical advantage of being able to detect small and / or distant objects, such as ). Fig. 6 As illustrated by reference, detection of image tiles can facilitate such results.

[0246] More generally, in some embodiments, imaging device(s)( 210 ) is a human observer( 115 It is configured to observe objects at an observation resolution associated with ), thereby enabling the device(s) to replace or supplement the service of a human observer.

[0247] now Fig. 5 Note that FIG. 5 is a reduced-resolution image(s) according to some embodiments of the subject matter of the patent disclosed herein ( 500 An exemplary generalized expression of ) 500 ) provides a schematic example. Image(s)( 500 ) is captured image(s)( 400 It is generated by reducing the resolution of ) from an M×N pixel resolution to a lower m×n pixel resolution. For example, m < M and / or n < N.

[0248] Therefore, the image resolution reduction process is the captured image(s)( 400 Includes reducing the horizontal resolution and / or vertical resolution of ). In some examples, the image ( 500 ) is the original image at this institution ( 400 Degraded image having reduced resolution compared to ) 500 ) or degraded resolution image( 500 It is also referred to as ).

[0249] In one non-limiting example, a captured image of 6000×3376 pixels is reduced to 1280×1280 pixels. Thus, in some examples, the two dimensions are reduced by different scales. In other examples, the two dimensions are reduced by the same scale. The scale can be an integer scale or a non-integer scale.

[0250] This reduced-resolution image(s) is one or more neural networks, also referred to herein as reduced-resolution image neural networks(s). 327 It is transmitted as ). Reduced resolution images(s) using this neural network(s) 500 Object detection is performed on ). Objects are detected at reduced resolution. In some examples, each pixel of the image is represented as a feature in the NN.

[0251] In non-limiting examples of drawings, an image (located in or near a body of water) 400 Physical object captured in ) 630 , 640 , 643 , 645 , 650 , 653 , 655 , 657 ) is a detection object( 530 , 540 , 543 , 545 , 550 , 553 , 555 , 557 It is detected as ). Captured image( 400 The captured object of ) 660 and 665 Note that ) is not detected at reduced resolution. These are images( 400 Because it appears as a small object containing a relatively small number of pixels, the resolution is reduced, and the image( 500 If ) is derived, low-resolution neural network(s)( 327 These smaller objects cannot be detected using ). Similarly, in some cases, relatively small and / or nested objects ( 650 and 653 ) uses low-resolution neural network(s) due to relatively low resolution to handle a single object( 550 It is detected as ). Similarly, in some cases, adjacent (or very close) objects ( 640 and 643 ) uses low-resolution neural network(s) due to relatively low resolution to handle a single object ( 543 It is detected as ). The low-resolution detection process cannot distinguish them as two objects. This is an object ( 655 and657 The same applies to ).

[0252] The process disclosed above is at least a reduced resolution image on which this detection is performed ( 500 ) is the original image( 400 It is a low-resolution detection process in that it has a resolution lower than that of ).

[0253] Detected object detected at low resolution ( 530 ) is also referred to herein as a detection object detected at low resolution or simply a low-resolution detection object. In some examples, (e.g., the sea or ocean in the area is empty or the only object present is an image ( 500 If the object is too small to be detected in ), the object is not detected.

[0254] Some exemplary technical advantages in performing low-resolution (i.e., reduced resolution) detection processes are that they facilitate detection using standard, inexpensive, and / or relatively fast neural networks (NN). The use of inexpensive components can be advantageous, for example, in commercial / civil navigation applications. Image ( 400 ) is, at least Fig. 6 For reasons further disclosed herein with reference to [the source], it was captured at a relatively high resolution. However, the high-resolution image ( 400Object detection in ) will require high-resolution NNs. In some cases, NNs large enough to support such images do not exist at the current level of technology, or require special development efforts (rather than being readily available as standard, inexpensive products). In some cases, such NNs supporting high resolution are expensive and, in some instances, can consume a lot of power, for example, using a relatively large number of computer resources. Additionally, in some cases, detection using such large NNs requires a relatively long time, and in some cases, too long to be practically used in navigation and collision avoidance applications.

[0255] Therefore, image( 400 ) image( 500 Reducing to ) and performing detection on a relatively small, appropriately sized NN can enable some or all of these advantages. Detection on the entire captured image ( 400 Since it is performed on ), this detection is performed on multiple image tiles ( 601 A relatively large vessel or other object spanning ) 657 ) can detect ( Fig. 6 As further explained herein with reference to ). That is, the image ( 500 ) is the original image( 400 Although the resolution is lower than ) but the navigation body ( 110 ) provides the "big picture" of the surrounding environment.

[0256] In many embodiments, the resolution is reduced by a multiple of 2 at each dimension, for example, by a multiple of 2, 4, 16, etc.

[0257] now Figures 6a and 6b Note that FIGS. 6a and FIGS. 6b are image tiles according to some embodiments of the subject matter of the patent disclosed herein ( 600 An exemplary generalized expression of ) 600) schematically exemplifies. The drawing is one or more captured images ( 400 This illustrates performing a high-resolution detection process, i.e., an image tile-based detection process, by dividing ) into image tiles. Image tile set ( 601 , 602 etc.) are image(s)( 400 It is derived from or generated from one or more of ).

[0258] Fig. 6a 9 image tiles arranged in a 3×3 grid solely for illustrative purposes ( 601 , 602 , 603 , 604 , 605 , 606 , 607 , 608 , 609 ...is illustrated. In another non-limiting example, the tiles are arranged in groups of 5 horizontally and 3 vertically, making up a total of 15 tiles.

[0259] Each image tile of the set ( 601 ) is captured image(s)( 400 It corresponds to a subsection of ), meaning that the tile does not contain the entire captured image. For example, the tile ( 601 Image tiles such as ) are captured image(s)( 400 It includes fewer vertical pixels than the number of vertical pixels of ) and / or fewer horizontal pixels than the number of horizontal pixels of the captured image. That is, in one or both dimensions, the image tile has a second number of pixels, while the captured image has a first number of pixels, and the second number is less than the first number. That is, each image tile is the entire captured image ( 400 It does not include all of the pixels of ).

[0260] As a single exemplary example of this, the image ( 400 ) has a resolution of 6000×3900 pixels, whereas each image tile ( 601) consists of 2000×1300 pixels. In this case, image tiles ( 601 The pixel density of ) is the image( 400 Same as ), but image tiles are images( 400 A part of ), that is, the whole image ( 400 It contains fewer pixels because it represents only a subsection of the image, not the entire image. An image tile is an image( 400 It indicates the incomplete part of ). The numbers are presented for illustrative purposes only.

[0261] Image tiles ( 601 At least some of the resolutions of ) (also referred to herein as image tile resolution) are reduced resolution image(s) ( 500 It is higher than the reduced resolution of ). In some embodiments, the resolution of at least some of the image tiles (e.g., pixel density, i.e., pixels per inch or per centimeter) is higher than that of the originally captured image(s) ( 400 It is equivalent to the resolution of M×N. This special case can be referred to as a "full resolution" image tile.

[0262] In some other exemplary implementations, the image tile resolution of one or more image tiles is an image ( 400 It is reduced somewhat from the resolution of ). For example, the resolution can be reduced by 2%, 5%, or 10%. These are non-limiting examples. Image tile( 601 This relatively small reduction in resolution of ) is reduced resolution image(s)( 500 It is performed by an amount less than the reduction performed to acquire ). Image tile(s)( 601 The resolution of ) is still high enough to achieve the desired result. For example, reduced-resolution image tiles are buoys ( 665 It still maintains a resolution high enough to still detect distant and / or small objects, such as ).

[0263] In some examples, image tile sets ( 600 At least some adjacent image tiles of ) have at least partial horizontal overlap and / or at least partial vertical overlap. Therefore, for example, tile ( 608 )(illustrated with dashed lines) and tile( 609 )(illustrated with a dashed line) is a zone( 667 It has partial horizontal overlap as illustrated by ). Similarly, the tile ( 606 )(illustrated with thick dashed and dotted lines) and tiles( 609 ) is a zone( 627 It has partial vertical overlap as illustrated by ). Similarly, zone ( 663 ) consists of 4 nested image tiles ( 604 )(illustrated with a dashed line), ( 605 )(illustrated with a thick dashed line), ( 607 )(illustrated with thick dashed and dotted lines) and ( 608 Illuminates the overlapping region of ). Therefore, image tile( 600 ) is an image with optional overlap of tile boundaries ( 400 Part or whole image of ) 400 Covers )

[0264] In some examples, the image tiles of the set have at least 5% overlap. In some examples, the tiles have at least 10% overlap.

[0265] In some examples, at least some adjacent image tiles in an image tile set do not have horizontal and vertical overlap.

[0266] In some examples, at least some adjacent image tiles in a set are separated by gaps in the horizontal and / or vertical directions. For example, there are "gaps" of a relatively small number of pixels between adjacent tiles within a horizontal row and / or vertical column. Fig. 6b Referring to, for example, an alternative case of a tile arrangement ( 600' ) is exemplified. Image tile( 603')(illustrated with dashed lines) and tile( 602' )(illustrated with a dashed line) is a zone( 623 It is separated by a horizontal gap as illustrated by ). Similarly, the tile ( 601' )(illustrated with dashed lines) and tile( 604' ) is a zone( 620 It is separated by a vertical gap as illustrated by ).

[0267] For the convenience of explanation Fig. 6b Note that only some tiles are depicted in this.

[0268] In one non-limiting exemplary example, the rightmost pixel of the left tile is the captured image ( 400 It corresponds to pixel #500 out of the 1280 horizontal pixels of ), and the left pixel of the right tile is the captured image( 400 It is pixel #505 of ). Therefore, pixels #501 through 504 of the captured image are a set ( 600 It is not included in any derived image tile of ). This means that in some cases, even if the tile "skips" some pixels, it can still provide sufficiently good detection of objects with sufficient accuracy, etc.

[0269] Similarly, in some examples, at least some adjacent image tiles "contact" each other in the horizontal and / or vertical directions. Therefore, for example, Fig. 6b In, image tile( 605' and 606' ) are in contact with each other in the horizontal direction. That is, the right pixel of the left tile and the left pixel of the right tile are the originally captured image ( 400 It is two horizontally adjacent pixels on ). A tile consists of "consecutive" pixels. For example, a tile ( 605' The rightmost pixel of ) is the captured image( 400 While tile i of ) is, adjacent tile( 606' The leftmost pixel of ) is the image( 400 It is tile i+1 of ). Contacting tile( 605' ,606' The area within the boundary zone between ) is referred to herein as the contact area (not shown in the drawings for convenience of explanation).

[0270] Note that in the non-limiting examples in the drawings, the image tiles are not all the same size. In other examples, they have the same size.

[0271] Accordingly, there are a number of possible embodiments of image tiling, including those not disclosed herein.

[0272] In some examples, image tile sets ( 600 ) is one or more captured images ( 400 It corresponds to all M×N pixels of ). That is, the image tile set ( 600 ) is one or more captured images ( 400 Covers ). In another example, the captured image ( 400 Not all pixels of ) are represented on the image tile. One example of this is Fig. 6b In the example of, the gap ( 620 , 623 It is the existence of ). Pixels associated with the gap are not represented on any image tile. In such an embodiment, an image tile set ( 600' ) is captured image(s)( 400 It covers only a part of ).

[0273] Similarly, in some implementations, image tile(s)( 601 ) does not include all pixels within the "area" demarcated by that specific tile. As a non-limiting example presented solely for illustrative purposes, the captured image ( 400 Every 8th column and / or every 8th row of pixels of ) is an image tile ( 601Even if not represented in ), sufficient visual information is preserved to enable high-quality object detection in tiles, while allowing the use of relatively smaller NNs (compared to the size of NNs for image tiles that do not skip pixels / rows / columns). In other words, all numerical examples are provided for illustrative purposes only.

[0274] For example, multiple cameras at time t ( 226 and 228 In some such implementations, where multiple images are captured by the same camera at multiple times t, t+1, etc. by ) and / or multiple times t, t+1, etc., Fig. 6 Note that the image tiling process is not performed for all images on which detection is performed, but only for specific images.

[0275] In the second part of the high-resolution detection process, the captured image(s)( 400 ) image tile( 607 After splitting into ), the system( 300 ) is an image tile neural network(s)( 327 Using ) set( 600 ) image tile( 607 For ) object( 660 Performs tile-based detection of ). Thereby, the system performs one or more tile-based detection objects ( 680 Acquires ).

[0276] tile( 607 Detection for ) is image( 500 A resolution higher than the resolution of ), for example, a captured image ( 400 Because it was performed at a resolution of M×N or a resolution close to this, the system small buoy( 660 , 680 Note that it can detect ). This is a reduced resolution image ( 500 It could not be performed in ). On the other hand, the buoy( 660 Detection of ) 680) Full captured image( 400 Image that is not ) 400 A tile containing only a part of ) 607 Because it was performed in ), tile( 607 Smaller, cheaper, faster, and / or off-the-shelf neural network(s) scaled to fit a relatively small number of pixels of ) 327 It is possible to use ). In some examples, the entire image( 400 Using a NN scaled to fit the entire image( 400 It is not feasible to perform detection on ). Therefore, a small buoy was detected, and the detection was carried out in a relatively efficient manner.

[0277] Note that in some cases, detection of tile-based objects does not occur on all image tiles. In one example, detection does not occur on certain tiles because there is no vessel / object in that area at that time, or, in some cases, the image in which the tile was captured ( 400 This may be because it covers the part corresponding to the empty sky among ).

[0278] The final set of detected objects (not shown in the drawing) consists of zero or more detected objects ( 540 , 557 ) and zero or more tile-based detection objects( 670 , 677 It can be obtained by merging ). That is, the system ( 300 )silver Fig. 5 The process of and Fig. 6 It has information items (detection objects) obtained from both processes and forms a consistent overall picture based on the output of the two stages without duplication and without omission of items.

[0279] For example, merging is a detection object ( 557 and 667 Both ) are a single actual object( 657Determines that it corresponds to ), and therefore the object( 657 Only one version of the image (or other data) is stored, for example, in a list, displayed to the user, and / or forwarded to another system or process. Therefore, the merging process eliminates duplicates, that is, multiple detections of the same object.

[0280] Similarly, small objects ( 660 ) is an image( 500 Although not detected in ), the final set of detected objects is small objects ( 660 ) is a tile-based detection object( 680 Because it was detected as ), it is a small object( 660 Includes ). Fig. 5 and Fig. 6 The objects detected in each instance may also be referred to herein as candidate detection objects, whereas, for example, the final set of detection objects selected from among the candidates is the result of the merging process.

[0281] Some objects ( 657 , 630 , 602 ) is sufficiently large, and accordingly, some objects ( 657 , 630 , 602 )Is Fig. 6 Note that it appears in multiple tiles. In some examples, detection in each individual image tile is some object ( 657 , 630 , 602 Detects ) as a separate object. Therefore, for example, the object( 630 The part of ) consists of 4 tiles ( 604 , 605 , 607 , 608 4 different tile-based objects in each 690' , 690'' , 690''' , 690'''' It is detected as ).

[0282] In some cases, a single image ( 500Accurately detecting objects in ) is multiple images ( 690' , 690'' , 690''' , 690'''' It is easier than detecting parts of an object in ). In some cases, parts of an object are in multiple images ( 690' , 690'' , 690''' , 690'''' A single image ( 614 )at 640 and 643 It is easier to accurately perform segmentation of adjacent, contacting, and / or overlapping objects such as. In some examples, these objects (e.g., 650 and 653 ) is a single image tile( 604 Note that accurate segmentation of such objects is possible using image tiles when they appear within ).

[0283] Therefore, by merging the first information derived from a low-resolution but "big picture" image with the second information derived from a high-resolution image tile focused on a smaller portion of the captured image, it is possible to obtain both the advantage of accurately detecting large objects crossing image tile boundaries and the advantage of detecting smaller objects (e.g., objects captured from a greater distance) that can only be detected at higher resolutions.

[0284] Detection can be performed on images captured with the highest resolution devices possible for the technology of the time, enabling the detection of even small objects at long distances. State-of-the-art cameras are utilized. However, such high-resolution captured images ( 400 Detection for both the big picture view and the more detailed high-resolution view of ) can be performed using a neural network configured for fewer pixels than the number of pixels in the captured image, which improves performance, i.e., enables faster detection, and a relatively inexpensive neural network (327 It enables the use of )(e.g., off-the-shelf NNs), thereby using relatively small computing resources. Note that smaller NNs typically operate faster than NNs configured for a larger number of pixels. Therefore, the method presented in these figures (state-of-the-art cameras with resolutions higher than those that standard and inexpensive NNs of the time could handle) 210 It facilitates the use of ). Specific performance values ​​of the camera and NN will naturally vary over time due to technological advancements. In other words, the method enables rapid object detection at the maximum possible camera resolution using standard hardware and NN. The method achieves a balance between camera performance and NN performance.

[0285] In some exceptional cases, reduced resolution images ( 500 Detection can be performed in ), but image tile set ( 600 Detection may not occur in ). A single large object in the image( 500 Detected in ), and the image is exactly 2 adjacent image tiles ( 604 , 605 Consider a theoretical case that exactly fits ). In one of the image tiles, there is no background behind the object, and no detection occurs. In such a case, the tile-based detection object was not detected, and merging the two detections to generate the final set of detected objects is the detection object detected at low resolution ( 557 Only ) is used. In another case, the image captured at time t( 400 In ), only the sea and sky without objects are visible, and the image( 500 ) and tile set( 600 No objects were detected in both.

[0286] In some exceptional cases, the image ( 500 If no objects are detected in ), Fig. 6 Objects detected according to ( 677 ) becomes the final detection object set. Fig. 6 If no object is detected according to, the image ( 500 Objects detected in ) 557 ) becomes the final set of detected objects. The "true" merging of the two detections occurs when at least one object is detected in each of the reduced resolution detection method and the image tile-based detection method.

[0287] In many practical implementations, it is advantageous to include additional detection steps and, in some cases, merging steps in the above detection process. These additional steps are optional and may provide additional technical advantages in some cases.

[0288] Image tile set ( 600 One or more Local Query and Segmentation (LIAS) definition regions associated with ) 612 Regarding ), the system( 300 ) is at least one local lookup neural network ( 237 LIAS is performed at a spatial resolution higher than the reduced resolution by utilizing ). In some examples, the segmentation process uses methods known in themselves. The local lookup-based detection object set is merged with the image tile-based detection object set. Therefore, with the help of these additional processes, the updated or modified final detection object set ( 690 ) can be obtained.

[0289] Merging eliminates duplicate detection of identical objects. This can yield an improved set in that it eliminates duplicate detection and / or provides more specific segmentation of adjacent, contacting, and / or overlapping objects. In some cases, parts of an object are in multiple images ( 604 , 605 and 606 A single image ( 614It is easier to accurately perform segmentation of such objects in ).

[0290] Fig. 7 As disclosed by reference, the merging of various detection objects can be performed in various orders.

[0291] The spatial resolution used in the LIAS method is referred to herein as LIAS resolution or local lookup resolution. In some examples, this refers to image tiles ( 601 It is equal to the image tile space resolution associated with one or more of ). In some examples, the LIAS method is the captured image ( 400 It uses full resolution detection performed at the resolution (pixel density) of ). This method is also referred to herein as the LIAS method.

[0292] In some implementations, definition scope(s)( 612 ) indicates the detection of one or more tile-based detection objects on more than one image tile. In some examples, the LIAS defined region(s) are the overlapping regions of the image tiles ( 627 , 667 Represents the detection of tile-based detection object(s) in ). Therefore, the object( 630 )Is ( 690' ), ( 690'' ), ( 690''' ), ( 690'''' It was detected as ). Similarly, the object( 645 ) is tile( 603 ) and tile( 606 ) appears in both. Such an area is also referred to herein as an overlapping detection area. Overlapping may occur between horizontally adjacent tiles and / or vertically adjacent tiles.

[0293] In another example, the definition area(s) represent the detection of tile-based detection object(s) across the tile boundaries, and do not necessarily represent detection in the overlapping area. Therefore, for example, the object( 657 ) is tile( 604, 605 and 606 It appears in all of ), and its detection ( 677 ) is formed across multiple tile boundaries. The definition area is also referred to herein as the LIAS definition area, local search area, or LIAS area.

[0294] Therefore, the LIAS method can be used to clarify the situation at tile boundaries and overlaps.

[0295] In some implementations, LIAS definition area ( 612 , 618 ) is a part of at least two image tiles, for example, a tile ( 604 , 605 , 607 and 608 A part or tile of ) 604 , 605 and 606 It includes the part of ), that is, the part Fig. 6a It includes a plurality of partial image tiles as exemplified in [Figure]. In the example of the drawing, the definition area is the intersection of at least two image tiles. In some embodiments, the definition area ( 612 ) is any image tile( 604 It is not identical or consistent with ).

[0296] In another implementation (not shown), the LIAS defined area is one or more full image tiles, for example, a tile ( 601 and 602 It includes the entirety of ). This means, for example, that the definition area is image tiles ( 601 and 602 It contains more pixels than any of ), but the number of pixels within the defined area (captured image( 400 This can occur when the number of pixels is relatively small compared to the total number of pixels within, and is small enough to require a fast / cheap neural network.

[0297] For example, object( 640 and 645) also appears on more than one tile, and the object( 640 and 645 Note that a definition area may also be defined around ). However, this area is not shown in the drawing solely for the convenience of explanation.

[0298] In some implementations, the LIAS process is one or more LIAS regions ( 612 At least one additional image tile corresponding to ) 614 It includes generating ) and performing local search within additional image tile(s). These additional image tiles are specifically defined for search. These image tiles are LIAS, local lookup, or local search image tiles (in this invention) 614 It is also referred to as ). For example, LIAS image tile( 614 ) is more than one image tile( 604 in ) or more than one image tile( 604 Objects that appear to have been detected across ) 630 An image captured around a pixel believed to contain ) 400 Defined based on ). LIAS image tile(s)( 614 ) is a local lookup neural network ( to perform local lookup detection 327 It is supplied to ).

[0299] In some cases, multiple images ( 604 , 605 , 607 , 608 Rather than detecting parts of an object in a single image ( 614 It is easier to accurately detect objects in ). Image( 614 ) maintains a sufficiently high resolution for the necessary detection requirements while simultaneously LIAS NN( 327 While keeping ) within the desired size, the entire object( 630It can be selected in a size that can include ). In other words, in some implementations, the use of standard hardware and networks provides fast detection at full resolution along with segmentation.

[0300] Local search is the same NN used for image tile-based detection ( 327 You can utilize ), utilize different NNs of the same size, or even utilize NNs of different sizes.

[0301] In some cases, objects ( 657 ) is relatively large and has a relatively large number of tiles ( 604 , 605 , 606 It crosses the boundary of ). Therefore, the local search area containing the large object ( 618 ) also has a relatively large size. In some examples of such cases, the system has such an area ( 618 Without performing LIAS on ), instead reduced resolution image( 500 It can be decided to rely on whole-image-based detection in ). Such a decision is based on the defined region ( 618 ) may be removed if it requires an excessively large neural network that exceeds the capabilities of the LIAS subsystem and / or has a size that cannot achieve the desired performance.

[0302] Each tile captured image ( 400 Since it is defined based on the pixels of ), first, the detection object( 630 The tile corresponding to ) 604 , 605 , 607 and 608 Once the pixels of interest of ) are determined, these are the image( 400 It can be mapped to the corresponding pixel on ), and accordingly, for example, according to a method known itself, a local lookup image tile ( 614) can be defined. Pixel mapping of the same type can also be the merging of image tile-based detection objects and objects detected at reduced resolution, and image tile-based detection objects and one or more local lookup-based detection objects ( 690 It can also be used for merging ).

[0303] For example, image tile detection object ( 670 In contrast to, 680' and 685' Not only image tile detection objects such as, but also objects ( 650 and 653 Note that the detection of ) is fully detected in only one tile, and therefore there is no need to perform the LIAS process for them.

[0304] Fig. 6 It should be noted that, for clarity in the drawing description, reference numbers for each image tile-based detection object and LIAS-based detection object are not shown.

[0305] In another example, the method includes image tile-based detection, LIAS-based detection, and the merging of the two detections.

[0306] In some exemplary implementations, Fig. 4 , Fig. 5 , Fig. 6 and Fig. 7 The method disclosed with reference to is performed in real time. Therefore, in some examples, image(s)( 400 ) is captured in real-time, and the system( 300 ) is configured to perform the detection method in real time. In some examples, this is a surface vessel ( 100 ) and objects( 133 , 140 It can facilitate collision avoidance between one or more of the methods. Therefore, the real-time or online nature of the method can provide advantages in real-life situations such as collisions.

[0307] Therefore, in some examples, the system ( 300) is configured to perform the method within 0.01 seconds to 60 seconds. In another example, the system is configured to perform the method within 0.01 seconds to 1 second. In yet another example, the system is configured to perform the method within 1 second to 60 seconds. In yet another example, the system is configured to perform the method within 10 seconds to 90 seconds. Such time frames may, in some cases, enable a surface vessel to respond within 1 minute to 15 minutes. Surface vessel ( 110 Based on the distance between ) and a specific object, their speeds, and their direction of travel, longer or shorter response times are required to avoid collisions or, for example, to change direction in an accurate manner. These figures are all non-limiting examples.

[0308] In another embodiment, the method is performed in near real time.

[0309] In some cases, specific engineering choices made when designing the method (e.g., steps and their sequence) and the equipment / components used are an attempt to achieve optimizations in terms of at least the time required for execution, computing costs, and sufficiently good performance. That is, in many cases, it is necessary to accurately and reliably detect even small and / or distant objects (and, where possible, classify, identify, and / or determine the location and size of these objects) in a rapid manner that enables timely response, while using limited and relatively expensive resources. Various types of detection and their merging can facilitate this optimized behavior.

[0310] now Fig. 7 Note that FIG. 7 shows a data flow according to some embodiment of the subject matter of the patent disclosed herein ( 700 An exemplary generalized expression of ) 700It schematically illustrates ). Rectangles represent actions, methods, processes, or associated systems, whereas parallelograms represent input / output data items.

[0311] An exemplary data flow is a block 710 Starting from, one or more images ( 400 ) is imaging device(s)( 210 It is captured by ). In this example, the high resolution of this image is 6000×3376. Note that all figures in this drawing are provided for illustrative purposes only.

[0312] First, considering the right side of the flow, block 720 In, image( 400 Reduced resolution image ( 500 ) is calculated. In this example, the reduced spatial resolution is 1280×1280.

[0313] image( 500 ) is a low-resolution neural network ( 327 ), for example, is transmitted to a deep neural network (DNN) 724 NN's output( 728 ) is a detected object detected at reduced resolution ( 557 This is Set #1 of ).

[0314] Now, considering the left side of the flow, the block 730 In, the captured image ( 400 ) is one or more image tiles ( 601 It is divided into ). In the example, the division consists of 15 tiles of size 1280×1280 pixels, and these tiles are arranged, for example, in a 5×3 arrangement. Output( 733 ) is a tile set( 600 )am.

[0315] The image tile(s) are high-resolution image tile neural network(s) 327 ), for example, is transmitted to a deep neural network ( 735 NN's output( 737) is a detected object detected at relatively high resolution ( 737 It is the second set #2 of )(tile-based detection objects).

[0316] block 740 In, the area for local lookup and segmentation ( 612 ) is defined. For example, LIAS image tile( 614 ) is defined. This refers, for example, to objects detected in more than one image tile ( 657 It is performed on ). Exemplary output( 745 ) is set #2 of detection objects combined with LIAS requests.

[0317] block 750 In this case, LIAS is performed on Set #2 upon the LIAS request. The LIAS image tile(s) are high-resolution local lookup and segmentation neural network(s) 327 ), for example, is transmitted to a deep neural network ( 750 NN's output( 755 ) is the third set #3 of detection objects (local lookup-based detection objects) detected at such relatively high resolution.

[0318] block 760 In this case, detection object #3 and detection object #2 are merged. For example, the merged set #4 of the objects detected at high resolution is output ( 765 )am.

[0319] block 770 In this case, detection object set #4 and low-resolution detection object #1 are merged. The merged detection object set #5 is output ( 780 )am.

[0320] Exemplary data flow ( 700 Note that ) initiates three different detections and two merges. In a non-limiting example of the drawing, the order of merging is to first merge the results of image tile detection and local search, and then the output of this merge is the image ( 500It involves merging with the result of reduced resolution detection for ). In other examples, the order of merging is reversed. Other merging orders and combinations are possible. For example, in one implementation, three outputs are all merged simultaneously as three parallel inputs.

[0321] Accordingly, depending on a specific selected implementation, one merging process may produce a "final" (or "intermediate") set of detection objects, and a second merging process may modify said set to produce an "updated" or "modified" set of final detection objects.

[0322] In some embodiments, a low-resolution detection process (block 720 , 724 , 728 ) is an image tile( 601 It is performed prior to the high-resolution (i.e., image-based) detection process associated with ). In other examples, the order is reversed. In other cases, the processes are performed at least partially in parallel, for example, simultaneously. In some implementations, this is an engineering decision to balance the number / amount of computer resources utilized with the total process execution time.

[0323] To exemplify these extensive options, Fig. 7 ... exemplifies a non-limiting example in which LIAS detection occurs after image tile-based detection, and these two detections are performed in parallel with low-resolution image detection. On the other hand, the method disclosed below Fig. 8 It exemplifies a non-limiting example of three detections, all performed in a serial manner: low-resolution detection, followed by tile-based detection, and then local lookup-based detection.

[0324] FIGS. 8a to 8c With reference to FIGS. 8a through 8c, a generalized flowchart of the flow of a process or method for object detection according to some embodiments of the subject matter of the patent disclosed herein ( 800 ) provides a general example. This detection process (800 ) in some examples Fig. 2 , Fig. 3a and Fig. 3b It is performed by a system such as the disclosed system by referring to . flow( 800 )silver Fig. 8a block of 803 It starts from.

[0325] According to some examples, one or more images ( 400 ) is captured (block 803 ). In some examples, this involves one or more imaging devices ( 210 ), for example, is performed by a camera. In some examples, the camera is a processing circuit ( 320 Captured image interface and control module of ) 333 Using ) the system( 300 It is controlled by ), for example, an I / O module ( 380 ) and communication networks( 247 It is interfaced through ).

[0326] According to some examples, one or more captured images ( 400 ) is received (block 807 ). In some examples, this is the captured image interface and control module ( 333 It is performed by ), for example, an I / O module ( 380 ) and communication networks( 247 It is interfaced through ).

[0327] According to some examples, imaging device(s)( 210 The input from ) is the IMU( 242 )(or INS( 240 Synchronized with input from )) (block 809 ). In some examples, this is an input / output module ( 380 It is performed by utilizing ), or by some other modules (e.g., dedicated modules not shown in the drawings), for example, a communication network ( 247It interfaces with the INS / IMU using ). This synchronization information is, for example, IMU readings and accurate images captured at the same time or nearly the same time ( 400 It can facilitate associating with ) over time.

[0328] According to some examples, image resolution reduction is performed (block 812 ). For example, a captured image ( 400 The spatial resolution of ) is reduced, and one or more reduced resolution images ( 500 ) is, for example Fig. 5 and Fig. 7 It is generated using the method disclosed by referring to . In some examples, this is a resolution reduction module ( 335 It is performed by ). In another example, resolution reduction is performed by an external system (not shown).

[0329] According to some examples, relatively low-resolution neural network(s) 327 Reduced resolution image(s) using ) 500 Object detection is performed on ) (block 815 ). In some examples, this is, for example Fig. 5 and Fig. 7 Reduced resolution detection module (using the method disclosed by reference) 337 It is performed by ). One or more detected objects ( detected at reduced resolution) 557 ) is obtained.

[0330] According to some examples, image tiles ( 601 ) set( 600 ) at least one captured image ( 400 Derived from ) (block 820 ). Image(s)( 400 ) is, for example Fig. 6 and Fig. 7 It is divided into image tiles using the method disclosed by referring to [the reference]. Each image tile of the set ( 601 ) is a captured image( 400It corresponds to a subsection of ). In some examples, this is a tile forming module ( 340 It is performed by ). In another example, the formation or generation of image tiles is performed by an external system (not shown).

[0331] According to some examples, relatively high-resolution neural network(s)( 327 Using ) set( 600 ) image tile( 601 , 602 Tile-based object detection is performed on ) (block 825 ). In some examples, this is, for example Fig. 6 and Fig. 7 Using the method disclosed with reference to, a tile-based detection module ( 345 It is performed by ). One or more tile-based detection objects detected at high resolution ( 677 ) is obtained.

[0332] The flow is A through Fig. 8b block of 827 It continues.

[0333] According to some examples, images ( 400 One or more LIAS definition regions on ) 612 ) is defined (block 827 These local inquiry areas ( 612 ) is associated with a set of image tiles. This is, for example, Fig. 6 and Fig. 7 It is performed using the method disclosed by referring to ). The definition area is one or more image tiles ( 605 Tile-based detection object(s) in ) 677 This indicates the detection of ). In some examples, this step is the local search module ( 348 It is performed by ).

[0334] According to some examples, one or more LIAS definition regions ( 612 At least one additional image tile corresponding to ) 614 ) is generated (block 830). In some examples, this is, for example Fig. 6 and Fig. 7 Using the method disclosed by referring to , the local search module ( 348 It is performed by ).

[0335] According to some examples, at least one local lookup neural network with relatively high resolution ( 327 Local lookup and segmentation (LIAS), i.e., local search, is performed at a resolution higher than the reduced resolution by utilizing ) (block 832 This LIAS process involves performing local lookup detection. In some examples, this is, for instance, Fig. 6 and Fig. 7 Using the method disclosed by referring to , the local search module ( 348 It is performed by ). One or more local query-based detection objects detected at high resolution ( 690 ) is obtained.

[0336] According to some examples, local lookup detection object(s) and tile-based detection object(s) are merged (block 840 ). Merging is a block 825 and 832 It targets objects detected in. In some examples, this is, for instance, Fig. 6 and Fig. 7 Using the method disclosed by referring to , the result merging module ( 350 ) or local search module( 348 It is performed by ). A merged set of detected objects detected at high resolution is obtained.

[0337] According to some examples, block 840 The merged set of detected object(s) from and the object(s) detected at reduced resolution (i.e., the low-resolution-based set) are merged (block 841 ). In this drawing example, the merge is a block 815 and blocks 840 It targets objects detected in. In some examples, this is, for instance, Fig. 6 and Fig. 7Using the method disclosed by referring to , the result merging module ( 350 It is performed by ). In some examples, the final set of detected objects or the updated final set is obtained.

[0338] According to some examples, the final set of detected objects(s) is merged with additional final set of detected objects(s) associated with different field of view (FOV) of the imaging device(s) (block 842 ). In this drawing example, merging blocks for two different FOVs 841 It targets objects detected in. In some examples, this is the result merging module ( 350 It is performed by ).

[0339] Therefore, the low resolution of the first FOV ( 530 ), tile-based( 690' , 690'' , 690''' , 690'''' ) and LIAS-based( 690 The detected object is merged with the corresponding object associated with the second adjacent FOV. For example, Fig. 2 The first camera of ( 226 FOV captured at time t in a specific field of view by ) A A first image including ( 400 ) and the second camera ( 227 Adjacent FOVs captured at the same time t by ) B A second image including ( 400' Consider )(not shown in the drawing). Physical object( 645 If ) is detected in both images, it is technically advantageous to merge the two detections to remove duplicates.

[0340] 2 FOVs A and FOV B is different. In certain cases of these non-limiting examples, the two FOVs have partial overlap. In some examples, the object ( 645 ) appears in the FOV nested area. In other cases, the object (645 ) is relatively large, and like Fig. 6 object of ( 657 Just as ) crosses the boundaries of the three image tiles, it crosses the boundaries of the FOV. Fig. 2 As disclosed above with reference to, in other examples, adjacent FOVs may touch at their edges / boundaries or even have a gap between them, and cameras such as pan / tilt and gimbal cameras may be utilized to facilitate unequal FOVs.

[0341] In some examples, merging is performed separately by detection type and then the results are merged. Therefore, the process may include at least one of the following.

[0342] (a) The low-resolution detection process includes merging the detection object(s) detected at reduced resolution in a first image captured at a first angle FOV of the imaging device(s) and additional detection object(s) detected at reduced resolution in a second image captured at a second angle FOV of the imaging device(s) that is not the same.

[0343] (b) The high-resolution tile-based detection process includes merging additional tile-based detection objects and tile-based detection object(s). Tile-based detection object(s) are associated with a set of image tiles captured in the first FOV. Additional tile-based detection objects are associated with additional image tiles captured in the second FOV.

[0344] (c) The high-resolution local lookup-based detection process includes merging additional local lookup-based detection objects and local lookup-based detection object(s). The local lookup-based detection object(s) are associated with an image tile captured in the first FOV. The additional local lookup-based detection object is associated with an additional image tile captured in the second FOV.

[0345] In this example, the three outputs of these three mergers are then merged to obtain a final set of detected objects that consider multiple FOVs / multiple cameras.

[0346] In other non-limiting examples, merging is performed separately by FOV. Therefore, the image associated with the first FOV ( 400 For ), merging of reduced resolution-based, tile-based, and LIAS-based detection objects is performed (in any order). The image associated with the second FOV ( 400' The same process is repeated for ). Then, the two outputs are merged. Therefore, the order and combination of merges can have various implementation examples.

[0347] According to some examples, the spatial locations of the detection objects in the final set are classified (block 847 ). In some examples, this is a classification and identification module ( 358 by ) or Fig. 3b It is performed by other modules not listed therein, for example, using methods known in themselves. In some examples, the location of the detected object is the captured image ( 400 At least based on the relative position of the detection object with respect to ) and the water platform associated with the same captured image ( 110 At least based on the orientation of ), that is, the image ( 400 It is determined based on the orientation of the platform at the capture time t of ).

[0348] The flow is B through Fig. 8c block of 850 It continues as.

[0349] According to some examples, the detection objects of the final set are classified (block 850 ). In some examples, this is a classification and identification module ( 358This is performed by, for example, using a method known in itself. For example, relatively common classes, such as aircraft carriers, oil tankers, fishing boats, leisure yachts, buoys, etc., are determined. This step is optional and is performed only in some implementations.

[0350] According to some examples, the detection objects of the final set are identified (block 854 ). In some examples, this is a classification and identification module ( 358 This is performed by, for example, using a method known as itself. For example, a more specific class, e.g., "XXXX-class aircraft carrier," is identified, or a specific vessel, e.g., "USS YYY" is identified. This step is optional and is performed only in some implementations.

[0351] Note that in some implementations or applications, such as the basic case of collision avoidance, classification or identification is not required. In some other examples, only some of the objects are classified and / or identified. Similarly, in some other examples, classification and / or identification is performed, for example, on objects detected at reduced resolution and / or on objects detected using image tiles, etc., at a stage prior to the generation of the final set of detected objects.

[0352] According to some examples, information representing the final set of detected objects is output (block 857 ). In some examples, this is an input / output module ( 380 It is performed by ), and the input / output module ( 380 ) is a communication network ( 247 The output is transmitted to a display or an external system (not shown) via ). In some examples, the output information includes location information (e.g., longitude and latitude coordinates) associated with one or more detection objects of the final set.

[0353] Other information that may be transmitted in some embodiments is, among other things, a surface vessel ( 110Object for ) 120 ) orientation, captured image of the object, estimated bounding box (e.g., size in pixels), estimate of object size (e.g., height), surface vessel ( 110 Includes an estimate of the distance from ) to the object.

[0354] The output may be, for example, in the form of a list of objects and parameters / information and / or a display form.

[0355] According to some examples, the method is repeated at least once more (block 870 ). Block 805 inner block 857 All or part of is repeated. In some embodiments, this is for a surface vessel (for example, to ensure collision-free, e.g., safe navigation) 110 It is repeated in a continuous manner throughout the entire operation period of the vessel. In some examples, the method is also performed repeatedly even while the vessel is anchored, providing continuous monitoring of the nearby waters environment. In some examples, this involves a repetition and tracking module ( 355 It is performed by utilizing ).

[0356] According to some examples, tracking of the detected object is performed (block 875 ). This tracing can be facilitated by iteration. In some examples, this involves iteration and tracing modules ( 355 This is performed using ). The continuous identity of a specific detection object is identified and tracked over time (e.g., in consecutive frames) not only within the field of view of a single imaging device but also across multiple FOVs of multiple devices, and the trajectory of the detection object can be evaluated. This can be performed for multiple detection objects. This step is step 870 Accordingly, the detection process can be performed continuously as it is repeated over time.

[0357] In some implementations, based on tracking, the system ( 300) can provide at least specific exemplary technical advantages. For example, an object ( 127 The speed and direction of travel of ) can be estimated. In addition, multiple images captured over time ( 400 If a relatively large change is detected between ), an alarm indicating that there may be a problem may be transmitted, for example, to an external system and / or a visual (and / or auditory) display. An example of such a situation is a large vessel ( 133 ) disappears in every image, or the object( 127 ) suddenly the ship( 110 Moving toward ) or the object( 127 There are cases where sudden acceleration occurs (for example, when an object appears to be moving slowly like a fishing boat, but then moves at the speed of a speedboat). Additional steps related to determining the change in the detected object position are disclosed in the following block.

[0358] Exemplary benefits of tracking are as follows: Water objects such as vessels ( 657 ) moves relatively slowly. Therefore, a high-resolution camera ( 210 ) images at a relatively slow frequency (e.g., 2 seconds) 400 It can be utilized to repeatedly capture ), and this can be used for the system ( 300 It will not overload my image processing pipeline. This can facilitate tracking objects relatively easily with high accuracy using high-resolution cameras. In some cases, such a slow image capture frequency is less suitable for images captured by airborne vehicles, such as aircraft.

[0359] Tracking is a water navigation vehicle ( 110As it navigates, it can provide continuous situational awareness. Furthermore, monitoring images over time can facilitate the detection of false alarms in some cases. Non-limiting examples of false alarms include cases where an object appears to be present at time t but is not visible at time t+1, for example, when what was seen at time t was actually reflections of the sun.

[0360] The flow terminates. As indicated above, a sequence of some or all of these steps may continue within a loop in some cases.

[0361] Accordingly, according to the flowchart, the subject matter of the patent disclosed herein includes a computerized object detection method, the method is performed by a processing circuit of an object detection system, and the method is,

[0362] A) Receiving one or more images captured by at least one imaging device mounted on a water platform -

[0363] One or more images include one or more objects located within a body of water -;

[0364] B)

[0365] i) Performing a high-resolution detection process based on dividing one or more images into image tiles - the high-resolution detection process is,

[0366] (1) Receiving a set of image tiles derived from at least one of one or more images -

[0367] Each image tile in the set corresponds to a subsection of at least one image, and

[0368] The spatial resolution of each image tile is higher than the reduced spatial resolution of one or more reduced resolution images -; and

[0369] (2) Performing tile-based object detection on a set of image tiles using at least one image tile neural network to obtain one or more tile-based detection objects -;

[0370] ii) For one or more defined regions associated with an image tile set, performing Local Query and Segmentation (LIAS) at a resolution higher than the reduced resolution using at least one Local Query Neural Network to acquire one or more Local Query-based detection objects -

[0371] One or more LIAS defined regions represent the detection of one or more tile-based detection objects in more than one image tile, and

[0372] Each LIAS defined area includes a portion of at least two image tiles -; and

[0373] iii) A step of performing a process including merging at least one of an image tile-based detection object set and one or more detection objects detected at reduced resolution with one or more local lookup-based detection objects to obtain a final detection object set.

[0374] Similarly, according to the flowchart, the subject matter of the patent disclosed herein includes a computerized object detection method performed by a processing circuit portion of an object detection system, and the method is,

[0375] A) Receiving one or more images captured by at least one imaging device mounted on a water platform -

[0376] One or more images include one or more objects located within a body of water -;

[0377] B)

[0378] i) Performing a low-resolution detection process - The low-resolution detection process is,

[0379] 1. Receiving one or more reduced-resolution images generated by performing image resolution reduction, which includes reducing the spatial resolution of one or more images; and

[0380] 2. Includes performing object detection on one or more reduced-resolution images using a neural network to acquire one or more detected objects detected at the reduced resolution.

[0381] ii) Performing a high-resolution detection process based on dividing one or more images into image tiles - the high-resolution detection process is,

[0382] (1) Receiving a set of image tiles derived from at least one of one or more images -

[0383] Each image tile in the set corresponds to a subsection of at least one image, and

[0384] The spatial resolution of each image tile is higher than the reduced spatial resolution of one or more reduced resolution images -; and

[0385] (2) Performing tile-based object detection on a set of image tiles using at least one image tile neural network to obtain one or more tile-based detection objects -;

[0386] iii) For one or more definition regions associated with an image tile set, performing Local Query and Segmentation (LIAS) at a resolution higher than the reduced resolution using at least one Local Query Neural Network to acquire one or more Local Query-based detection objects -

[0387] One or more definition regions represent the detection of one or more tile-based detection objects in more than one image tile, and

[0388] Each LIAS defined area includes a portion of at least two image tiles -; and

[0389] iv) By merging one or more detection objects detected at reduced resolution with a set of image tile-based detection objects and one or more local lookup-based detection objects,

[0390] It includes a step of performing a process that includes obtaining a final set of detected objects.

[0391] Therefore, the method is,

[0392] (A) Low-resolution detection and image tile-based detection (i.e., high-resolution detection process),

[0393] (B) Image tile-based detection and local lookup-based detection, or

[0394] (C) All three can be combined: image tile-based detection, local lookup-based detection, and low-resolution detection.

[0395] Any of these methods can yield a final set of detected objects.

[0396] In some embodiments, one or more steps of the flowcharts illustrated herein may be performed automatically. The flows and functions illustrated in the flowcharts are, for example, a system ( 300 ) and processing circuit section( 320 It can be implemented in, Fig. 2 , Fig. 3a and Fig. 3b The components described in relation to can be utilized. The flowchart is, for example, a system ( 300 ) and processing circuit section( 320 It is also noted that although the description refers to system elements that realize the steps as described above, this is by no means limited thereto, and the operation may be performed by elements other than those described herein.

[0397] It should be noted that the teachings of the subject matter of the patent disclosed herein are not bound by the flowcharts illustrated in the various drawings.

[0398] For example, some of the actions or steps may be combined into a single integrated action or separated into multiple actions, and / or other actions may be added. As a non-limiting example, in some cases, a block 840 , 841 and 842 It can be combined.

[0399] In the embodiments of the subject matter disclosed herein, fewer, more, and / or different steps may be performed compared to the steps illustrated in the drawings. As one non-limiting example, a specific embodiment is a block 875 It may not include one or more of.

[0400] Similarly, in some embodiments, operations may occur differently from the order illustrated. One or more stages illustrated in the drawings may be executed in a different order and / or one or more groups of stages may be executed simultaneously. As one example, block 812 and 815 is a block 820 to 840 It can be performed before or after.

[0401] In the following claims, the alphabetic characters and Roman numerals used to designate claim elements, such as components and steps, are provided for convenience only and do not imply any specific order in which the steps are performed.

[0402] It should be noted that the term "comprising" used throughout the appended claims should be interpreted to mean "comprising but not limited to."

[0403] Although examples have been presented and disclosed in accordance with the subject matter of the patent disclosed herein, it will be recognized that many modifications can be made without departing from the spirit of the subject matter of the patent disclosed herein.

[0404] It should be understood that the subject matter disclosed herein is not limited in its application to the details included herein or illustrated in the drawings. Other embodiments of the subject matter disclosed herein are possible and may be practiced and performed in various ways. Accordingly, it should be understood that the language and terminology used herein are for illustrative purposes only and should not be construed as limiting. Accordingly, those skilled in the art will recognize that the concepts on which this disclosure is based can be readily utilized as a standard for designing other structures, methods, and systems to achieve various purposes of the subject matter disclosed herein.

[0405] Additionally, it will be understood that a system according to the subject matter disclosed herein may be, at least in part, a suitably programmed computer. Likewise, the subject matter disclosed herein considers a computer program product readable by a machine or computer for executing the method of the subject matter disclosed herein, or any part thereof. The subject matter disclosed herein further considers a non-transient machine-readable memory or computer-readable memory that embodies in a tangible form a program of instructions executable by a machine or computer for executing the method of the subject matter disclosed herein, or any part thereof. The subject matter disclosed herein further considers a non-transient computer-readable storage medium embodied therein a computer-readable program code configured to be executed to perform the method of the subject matter disclosed herein.

[0406] Those skilled in the art will readily recognize that various modifications and changes may be applied to the embodiments of the present invention as described above, without departing from the scope of the appended claims and the scope defined by those claims.

Claims

Claim 1 A computerized object detection method performed by a processing circuit of an object detection system, comprising: A) receiving one or more images captured by at least one imaging device mounted on a surface platform—the one or more images include one or more objects located within a body of water—; B) i. performing a low-resolution detection process—the low-resolution detection process comprises, 1. Receiving one or more reduced-resolution images generated by performing image resolution reduction, which includes reducing the spatial resolution of one or more of the above images; and 2. Performing object detection on one or more reduced-resolution images using a neural network to obtain one or more detected objects detected at the reduced resolution -; ii. Performing a high-resolution detection process based on dividing the one or more images into image tiles - the high-resolution detection process comprises, 1. Receiving a set of image tiles derived from at least one of the above one or more images - Each image tile of the above set corresponds to a subsection of at least one image, and The spatial resolution of each of the above image tiles is higher than the reduced spatial resolution of one or more of the above reduced resolution images -; and 2. A computerized object detection method comprising: performing tile-based object detection on the image tiles of the set using at least one image tile neural network to obtain one or more tile-based detection objects; and iii. performing a process comprising merging the one or more detection objects detected at the reduced resolution with the one or more tile-based detection objects to obtain a final set of detection objects. Claim 2 A computerized object detection method according to claim 1, further comprising: C) performing LIAS at a resolution higher than the reduced resolution using at least one local lookup neural network for one or more local lookup and segmentation (LIAS) definition regions associated with the image tile set to obtain one or more local lookup-based detection objects - wherein the one or more LIAS definition regions represent the detection of the one or more tile-based detection objects in more than one image tile, and each LIAS definition region includes a portion of at least two image tiles -; and D) merging at least one of the image tile-based detection object set and one or more detection objects detected at the reduced resolution with the one or more local lookup-based detection objects to facilitate the acquisition of an updated final detection object set. Claim 3 A computerized object detection method according to claim 1 or 2, further comprising the step of performing the image resolution reduction. Claim 4 A computerized object detection method performed by a processing circuit of an object detection system, comprising: A) receiving one or more images captured by at least one imaging device mounted on a surface platform—the one or more images include one or more objects located within a body of water—; B) i) performing a high-resolution detection process based on dividing the one or more images into image tiles—the high-resolution detection process comprises, (1) Receiving a set of image tiles derived from at least one of the above one or more images - Each image tile of the above set corresponds to a subsection of at least one image, and The spatial resolution of each of the above image tiles is higher than the reduced spatial resolution of one or more of the above reduced resolution images -; and (2) performing tile-based object detection on image tiles of the set using at least one image tile neural network to obtain one or more tile-based detection objects; ii) performing LIAS at a resolution higher than the reduced resolution using at least one local lookup neural network on one or more local lookup and segmentation (LIAS) definition regions associated with the set of image tiles to obtain one or more local lookup-based detection objects - the one or more LIAS definition regions represent the detection of one or more tile-based detection objects in more than one image tile, and each definition region includes a portion of at least two image tiles -; and iii) performing a process including merging the set of image tile-based detection objects and the one or more local lookup-based detection objects to obtain a final set of detection objects. Computerized object detection method comprising the step of performing a process. Claim 5 A computerized object detection method according to any one of claims 2 to 4, wherein the one or more LIAS definition regions represent the detection of the one or more tile-based detection objects in at least one of the overlapping region and contact region of an image tile. Claim 6 A computerized object detection method according to any one of claims 1 to 5, wherein the one or more images include one or more high-resolution images, and the at least one imaging device includes at least one high-resolution imaging device. Claim 7 A computerized object detection method according to any one of claims 1 to 6, further comprising the step of deriving the image tile set. Claim 8 A computerized object detection method according to any one of claims 1 to 7, wherein at least one image is captured in real time, and the system is configured to perform the method in real time; near real time; and online, thereby facilitating collision avoidance between the water platform and one of the one or more objects. Claim 9 A computerized object detection method according to claim 8, wherein the system is configured to perform the method within 0.01 seconds to 1 second. Claim 10 A computerized object detection method according to claim 8, wherein the system is configured to perform the method within 1 second to 60 seconds. Claim 11 A computerized object detection method according to any one of claims 1 to 10, wherein the method is additionally repeated at least once. Claim 12 A computerized object detection method according to claim 11, wherein the system is configured to synchronize input from at least one imaging device with input from at least one of an inertial measurement unit (IMU) and an inertial navigation system (INS). Claim 13 A computerized object detection method according to claim 11 or 12, further comprising the step of performing tracking of one or more objects based on the above iteration. Claim 14 A computerized object detection method according to any one of claims 1 to 13, wherein the surface platform is one of a ship and a boat. Claim 15 A computerized object detection method according to any one of claims 1 to 14, wherein the final detection object set includes at least one of other water platforms, buoys, obstacles, and lighthouses. Claim 16 A computerized object detection method according to any one of claims 1 to 15, wherein the one or more captured images have a resolution of at least 640 pixels in at least one dimension. Claim 17 A computerized object detection method according to any one of claims 1 to 16, wherein the one or more captured images have a resolution of at least 1080 pixels in at least one dimension. Claim 18 A computerized object detection method according to any one of claims 1 to 17, wherein at least one of [a] at least some adjacent image tiles of the image tile set have at least partial horizontal overlap and / or at least partial vertical overlap; and [b] at least some adjacent image tiles of the image tile set are in contact with each other. Claim 19 A computerized object detection method according to any one of claims 1 to 18, further comprising the step of classifying at least one detection object of the final set. Claim 20 A computerized object detection method according to any one of claims 1 to 19, further comprising the step of identifying at least one detection object of the final set. Claim 21 A computerized object detection method according to any one of claims 1 to 20, wherein the at least one imaging device is included in one or more payloads, each payload includes one or more imaging devices, and the one or more payloads are configured to cover an angle of up to 360 degrees around the water platform. Claim 22 A computerized object detection method according to any one of claims 1 to 21, wherein at least one imaging device is a camera. Claim 23 A computerized object detection method according to claim 21 or 22, further comprising the step of merging an additional final detection object set with an additional final detection object set associated with image capture at a different field of view (FOV) of at least one additional imaging device. Claim 24 A computerized object detection method according to any one of claims 1 to 24, further comprising the step of K) outputting information representing the final detection object set. Claim 25 A computerized object detection method according to claim 25, wherein the output information includes location information associated with at least one detection object of the final set. Claim 26 A computerized object detection method performed by a processing circuit of an object detection system, comprising: A) receiving one or more images captured by at least one imaging device mounted on a surface platform—the one or more images include one or more objects located within a body of water—; B) i. performing a low-resolution detection process—the low-resolution detection process comprises, (1) Receiving one or more reduced resolution images generated by performing image resolution reduction including reducing the spatial resolution of one or more of the above images; and (2) Performing object detection on one or more reduced-resolution images using a neural network to obtain one or more detected objects detected at reduced resolution -; ii. Performing a high-resolution detection process based on dividing one or more images into image tiles - the high-resolution detection process is, (1) Receiving a set of image tiles derived from at least one of the above one or more images - Each image tile of the above set corresponds to a subsection of at least one image, and The spatial resolution of each image tile is higher than the reduced spatial resolution of the one or more of the reduced resolution images above -; and (2) performing tile-based object detection on image tiles of the set using at least one image tile neural network to obtain one or more tile-based detection objects; iii. performing LIAS at a resolution higher than the reduced resolution using at least one local lookup neural network on one or more local lookup and segmentation (LIAS) definition regions associated with the set of image tiles to obtain one or more local lookup-based detection objects - the one or more LIAS definition regions represent the detection of one or more tile-based detection objects in more than one image tile, and each LIAS definition region includes a portion of at least two image tiles -; and iv. performing a process including merging the one or more detection objects detected at the reduced resolution with the set of image tile-based detection objects and the one or more local lookup-based detection objects to obtain a final set of detection objects. Claim 27 A computerized object detection system comprising a processing circuit configured to perform the method of any one of claims 1 to 26. Claim 28 A computerized object detection system according to claim 27, further comprising at least one imaging device. Claim 29 A non-transient computer-readable storage medium that implements in a tangible form a program of instructions that, when executed by a processing circuit of an object detection system, cause the processing circuit to perform the method of any one of claims 1 to 26.

Images