Computer system and method for automated object detection in optical satellite imagery using machine learning

Abstract

Systems, methods, and storage media for automatically locating and characterizing ships in electro-optical satellite imagery are provided. The method includes: extracting a ground sample distance from metadata of an electro-optical satellite image; selecting a ship detection model from a plurality of ship detection models using the extracted ground sample distance; converting the electro-optical satellite image into a standardized format; processing the standardized image using the selected ship detection model to obtain a ship detection output including a bounding box for each ship detection; for each ship detection in the ship detection output, geocoding pixel coordinates of the bounding box into geocoded coordinates; generating a ship detection report including the geocoded coordinates of each ship detection in the ship detection output; and transmitting the ship detection report to a user computing device configured to display the ship detection report in a graphical user interface.

Description

COMPUTER SYSTEM AND METHOD FOR AUTOMATED OBJECT DETECTION IN OPTICAL SATELLITE IMAGERY USING MACHINE LEARNINGTechnical Field

[0001] The following relates generally to earth observation and surveillance, and more particularly to systems and methods for ship detection using computer vision techniques.Introduction

[0002] As satellite imagery becomes increasingly available, new approaches of obtaining intelligence from satellite imagery are desired. Obtaining intelligence can include the detection and tracking of objects in and across satellite images. One example application with particular value is the detection and tracking of ships, or dark vessels, across marine scenes captured in optical satellite images. Techniques are desired that enable automatic, fast, and accurate localization and classification of ships in optical satellite imagery of various resolutions.

[0003] Accordingly, there is a need for an improved system and method for ship detection and processing and analysis of optical satellite imagery that overcomes at least some of the disadvantages of existing systems and methods.Summary

[0004] A method of automatically locating and characterizing ships in electro- optical satellite imagery, the method comprising: storing, in a data storage device, an electro-optical satellite image comprising image metadata; extracting, using at least one processor, a ground sample distance from the image metadata; selecting, using the at least one processor, a ship detection model from a plurality of ship detection models each configured to detect ships in the electro-optical satellite image and localize each detected ship using a bounding box defined by bounding box coordinates, wherein the ship detection model is selected using the extracted ground sample distance; converting, using the at least one processor, the electro-optical satellite image into a standardized format (“standardized image”) for subsequent processing by the ship detection model; processing, using the at least one processor, the standardized image using the selectedship detection model to obtain a ship detection output in which each ship detection detected by the ship detection model is defined by a bounding box and an associated confidence score; for each ship detection in the ship detection output, geocoding, using the at least one processor, pixel coordinates of the bounding box into geocoded coordinates comprising latitude and longitude coordinates; generating, using the at least one processor, a ship detection report including the geocoded coordinates of each ship detection in the ship detection output; and transmitting, via a communication interface, the ship detection report to a user computing device configured to display the ship detection report in a graphical user interface.

[0005] The method may further comprise: for each ship detection in the ship detection output, extracting, using the at least one processor, a preview image comprising the corresponding bounding box enclosing the detection from the standardized image; for each preview image, processing the preview image using a ship characterization module to obtain a ship characterization output including a length and width of the ship detection, the ship characterization module configured to estimate the length and width from the dimensions of the bounding box enclosing the ship detection; wherein the ship detection report further includes the length and width of each ship detection in the ship detection output.

[0006] The selected ship detection model may be an oriented bounding box model, the ship characterization module may be further configured to estimate an orientation of the ship detection, and the ship detection report may further include the orientation of each ship detection in the ship detection output.

[0007] The method may further include: for each preview image, processing, using the at least one processor, the preview image using a ship type classification model to obtain a ship classification output including a predicted ship type for the ship detection; and wherein the ship detection report further includes the predicted ship type for each ship detection in the ship detection output.

[0008] The electro-optical satellite image may be in a GeoTIFF format with atmospheric compensation and orthorectification already applied.

[0009] The ground sample distance may describe a distance on ground between centers of each pixel.

[0010] Converting the electro-optical satellite image may include processing the electro-optical satellite image to have only red, green, and blue colour channels.

[0011] Converting the electro-optical satellite image may further include: where the electro-optical satellite image is panchromatic with only one channel, duplicating the one channel twice to obtain three identical channels, the three identical channels being the RGB channels; and where the electro-optical satellite image includes additional channels beyond the red, green, and blue channels, ignoring the additional channels.

[0012] The electro-optical satellite image may include a plurality of tiles, and converting the electro-optical satellite image may include stitching the plurality of tiles into a single geotiff image file.

[0013] The method may further include performing, using the at least one processor, spatial resampling on the standardized image to the same spatial resolution that the selected ship detection model was trained on.

[0014] The method may further include generating, using the at least one processor, a plurality of equally sized image chips from the standardized image, wherein the image chips are generated with some overlap, and processing the standardized image using the selected ship detection model may include processing the plurality of equally sized image chips using the selected ship detection model.

[0015] The plurality of ship detection models may include at least one horizontal bounding box (“HBB”) model and at least one oriented bounding box (“OBB”) model.

[0016] The plurality of ship detection models may include at least one oriented bounding box model.

[0017] The plurality of ship detection models may have been trained at different spatial resolutions, and the selected ship detection model and the standardized image may have the same spatial resolution, after the standardized image has been resampled to that spatial resolution.

[0018] The method may further include, for each ship detection in the ship detection output, geocoding, using the at least one processor, pixel coordinates of the bounding box into a geocoded detection comprising latitude and longitude coordinates, wherein the latitude and longitude coordinates are included in the ship detection output.

[0019] The method may further include filtering, using the at least one processor, the geocoded detections using an intersection over area (“loA”) metric to identify overlapping detections, determining that the overlapping detections have an loA meet an loA size threshold, and retaining only one of the overlapping detections with the largest area.

[0020] A computer system comprising at least one processor configured to execute any of the foregoing methods is also provided.

[0021] A non-transitory computer readable medium storing processor-executable instructions which, when executed by at least one processor, cause the processor to perform any of the foregoing methods is also provided.

[0022] A computer system for automatically locating and characterizing ships in electro-optical satellite imagery is also provided. The system includes a communication interface device for: receiving an electro-optical satellite image comprising image metadata; and transmitting a ship detection report to a user computing device configured to display the ship detection report in a graphical user interface. The system further includes a data storage device for storing: the electro-optical satellite image; and a plurality of ship detection machine learning models each configured to detect ships in the electro-optical satellite image and localize each detected ship using a bounding box defined by bounding box coordinates. The system further includes at least one processor configured to: extract a ground sample distance from the image metadata; select a ship detection model from the plurality of ship detection models, wherein the ship detection model is selected using the extracted ground sample distance; convert the electro-optical satellite image into a standardized format for subsequent processing by the ship detection model to obtain a standardized image; process the standardized image using the selected ship detection model to obtain a ship detection output in which each ship detection detected by the ship detection model is defined by a bounding box and an associatedconfidence score; for each ship detection in the ship detection output, extracting a preview image comprising the corresponding bounding box enclosing the detection from the standardized image; for each preview image, processing the preview image using a ship type classification model to obtain a ship classification output including a predicted ship type for the ship detection; for each preview image, processing the preview image using a ship characterization module to obtain a ship characterization output including a length and width of the ship detection, the ship characterization module configured to estimate the length and width from the dimensions of the bounding box enclosing the ship detection; and generating the ship detection report, the ship report including (i) the ship detection output and (ii) the ship classification and ship characterization outputs for each ship detection in the ship detection output.

[0023] A non-transitory computer readable storage medium storing processorexecutable instructions which, when executed by at least one processor, cause the at least one processor to perform a method of automatically locating and characterizing ships in electro-optical satellite imagery is also provided. The method includes storing, in a data storage device, an electro-optical satellite image comprising image metadata; extracting, using at least one processor, a ground sample distance from the image metadata; selecting, using the at least one processor, a ship detection model from a plurality of ship detection models each configured to detect ships in the electro-optical satellite image and localize each detected ship using a bounding box defined by bounding box coordinates, wherein the ship detection model is selected using the extracted ground sample distance; converting, using the at least one processor, the electro-optical satellite image into a standardized format (“standardized image”) for subsequent processing by the ship detection model; processing, using the at least one processor, the standardized image using the selected ship detection model to obtain a ship detection output in which each ship detection detected by the ship detection model is defined by a bounding box and an associated confidence score; for each ship detection in the ship detection output, extracting, using the at least one processor, a preview image comprising the corresponding bounding box enclosing the detection from the standardized image; for each preview image, processing, using the at least one processor, the preview image using a ship type classification model to obtain a ship classification output including apredicted ship type for the ship detection; for each preview image, processing the preview image using a ship characterization module to obtain a ship characterization output including a length and width of the ship detection, the ship characterization module configured to estimate the length and width from the dimensions of the bounding box enclosing the ship detection; generating, using the at least one processor, a ship detection report including (i) the ship detection output and (ii) the ship classification and ship characterization outputs for each ship detection in the ship detection output; and transmitting, via a communication interface, the ship detection report to a user computing device configured to display the ship detection report in a graphical user interface.

[0024] Other aspects and features will become apparent, to those ordinarily skilled in the art, upon review of the following description of some exemplary embodiments.Brief Description of the Drawings

[0025] The drawings included herewith are for illustrating various examples of articles, methods, and apparatuses of the present specification. In the drawings:

[0026] Figure 1 is a block diagram of a ship detection pipeline for detecting ships in optical satellite images, according to an embodiment;

[0027] Figure 2 is a block diagram of a computer system for automated ship detection in optical satellite images, capable of implementing the pipeline of Figure 1 , according to an embodiment;

[0028] Figure 3 is a block diagram of the satellite image metadata of Figure 2 in further detail, according to an embodiment;

[0029] Figure 4 is a block diagram of the ship detection module of Figure 2 in further detail, according to an embodiment;

[0030] Figure 5 is a block diagram of the detection inference model of Figure 2 in further detail, according to an embodiment;

[0031] Figure 6 is a block diagram of the inference data of Figure 2 in further detail, according to an embodiment;

[0032] Figure 7 is a block diagram of the post-processed image data of Figure 2 in further detail, according to an embodiment;

[0033] Figure 8 is a block diagram of the detections file of Figure 2 in further detail, according to an embodiment;

[0034] Figure 9 is a block diagram of the ship features determination module of Figure 2 in further detail, according to an embodiment;

[0035] Figure 10 is a block diagram of the predictions file of Figure 2 in further detail, according to an embodiment;

[0036] Figure 11 is a schematic diagram of a system for automated ship detection, according to an embodiment; and

[0037] Figure 12 is a block diagram of a computing device of Figure 11 , according to an embodiment.Detailed Description

[0038] Various apparatuses or processes will be described below to provide an example of each claimed embodiment. No embodiment described below limits any claimed embodiment and any claimed embodiment may cover processes or apparatuses that differ from those described below. The claimed embodiments are not limited to apparatuses or processes having all of the features of any one apparatus or process described below or to features common to multiple or all of the apparatuses described below.

[0039] One or more systems described herein may be implemented in computer programs executing on programmable computers, each comprising at least one processor, a data storage system (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device. For example, and without limitation, the programmable computer may be a programmable logic unit, a mainframe computer, server, and personal computer, cloud-based program or system, laptop, personal data assistance, cellular telephone, smartphone, or tablet device.

[0040] Each program is preferably implemented in a high-level procedural or object-oriented programming and / or scripting language to communicate with a computer system. However, the programs can be implemented in assembly or machine language, if desired. In any case, the language may be a compiled or interpreted language. Each such computer program is preferably stored on a storage media or a device readable by a general or special purpose programmable computer for configuring and operating the computer when the storage media or device is read by the computer to perform the procedures described herein.

[0041] A description of an embodiment with several components in communication with each other does not imply that all such components are required. On the contrary, a variety of optional components are described to illustrate the wide variety of possible embodiments of the present invention.

[0042] Further, although process steps, method steps, algorithms or the like may be described (in the disclosure and I or in the claims) in a sequential order, such processes, methods and algorithms may be configured to work in alternate orders. In other words, any sequence or order of steps that may be described does not necessarily indicate a requirement that the steps be performed in that order. The steps of processes described herein may be performed in any order that is practical. Further, some steps may be performed simultaneously.

[0043] When a single device or article is described herein, it will be readily apparent that more than one device I article (whether or not they cooperate) may be used in place of a single device I article. Similarly, where more than one device or article is described herein (whether or not they cooperate), it will be readily apparent that a single device I article may be used in place of the more than one device or article.

[0044] The following relates generally to earth observation and surveillance, and more particularly to systems and methods for automated object detection in optical satellite imagery using computer vision techniques. In an embodiment, the objects are ships.

[0045] While the present disclosure makes reference to automated detection, classification, and characterization of ships, it is to be understood that this is one particularexample implementation and that, in other embodiments, other types of objects may be detected, classified, and characterized using the systems and methods described herein.

[0046] The present disclosure provides a machine learning system that automatically locates and classifies ships in optical satellite imagery at various spatial resolutions. In an embodiment, the system may be deployed on the cloud as a standalone system.

[0047] The system of the present disclosure may provide automatic, fast, and accurate localization and classification of ships in optical satellite imagery of various resolutions.

[0048] The present disclosure provides a machine learning system that may outperform and thus replace traditional image processing workflows. In variations, the system uses novel deep learning models that are capable of leveraging large amounts of image data to make fast and accurate models for ship detection and classification.

[0049] In some embodiments, the automated ship detection system of the present disclosure may be integrated in or in communication with an earth observation surveillance software platform or application (e.g., a maritime intelligence insights software platform). A user may interact with such a system through a user device to order new optical imagery and have ships automatically detected in the images acquired and ingested into the platforms through operation of the automated ship detection system.

[0050] Referring now to Figure 1 , shown therein is a ship detection pipeline 100, according to an embodiment. The ship detection pipeline 100 may be implemented by one or more computing devices. The ship detection pipeline 100 may be encoded and stored in a non-transitory computer-readable storage medium as computer-executable instructions which, when executed by at least one processor, cause the computer to execute the pipeline. The pipeline 100 may be referred to as a ship detection software suite.

[0051] The ship detection pipeline 100 is an automated pipeline configured to ingest electro-optical imagery from one or more image sources and generate detailed ship detection reports. The detailed ship reports may be later ingested by a web-basedservice for providing maritime insights that combines multiple sources of information to detect dark vessels.

[0052] In an embodiment, the ship detection pipeline 100 ingests an image product 102 and processes the image product 102 using traditional image processing and deep learning components to automatically locate and characterize length, width, and (in some cases) orientation of one or more vessels within an image. The ship detection pipeline 100 produces a report 104 summarizing ship detections in the image 102. This may include a set of images cropped out of the original image 102, for easy viewing of the detected ships.

[0053] The pipeline 100 includes ship detection models. The ship detection models may be trained on large datasets by acquiring and labelling the high resolution imagery (e.g., 0.3-0.6 m / pix), and downsampling the datasets to other desired resolutions that match image source’s available imagery for operational use. Once model development is complete (e.g., architecture selection, training hyperparameter tuning, and performance evaluation), models are run in an inference mode in the ship detection pipeline 100.

[0054] In addition to multiple image resolutions, the ship detection pipeline 100 may include ship detection models that produce (i) horizontal bounding boxes around ships (“horizontal bounding box model” or “HBB model”), as well as ship detection models that produce (ii) oriented bounding boxes (“oriented bounding box model” or “OBB model”). Oriented bounding boxes may provide more precise length and width estimates of the detected ships, as well as their orientation.

[0055] In some embodiments, the ship detection models may include only HBB models or only OBB models. OBB models may require more detailed and precise labels for their training datasets, which can take much longer to produce. Once the OBB model is finished, however, the OBB model may provide improved performance over HBB models in crowded scenes (e.g., crowded port scenes). In such crowded port scenes, HBB models may produce HBBs of nearby ships with more overlap, which may result in them being discarded as potentially duplicate labels of the same ship. An OBB model may more provide accurate estimates of ship length and width and orientation (accuracymay be significantly better). However, in some cases, OBB models may have lower accuracy in detecting ships, meaning that the OBB model may miss more ships in general. Accordingly, there may be advantages to embodiments using only OBB models, only HBB models, or a combination of HBB and OBB models.

[0056] In an embodiment, the one or more horizontal bounding box (HBB) models use a RetinaNet architecture with some minor custom izations, and either EfficeintNetB2 or MobileNetV3 backbones.

[0057] In an embodiment, the one or more oriented bounding box (OBB) models use an Oriented-RCNN architecture with SWIN backbones.

[0058] In some embodiments, an operator user selects whether to run HBB or OBB mode, which dictates which type of model is used. The selection may be provided to the system via a graphical user interface displayed on a user device. HBB may provide slightly better performance (i.e. , fewer missed ships and false detections), while OBB may provide more accurate length and width estimation.

[0059] The ship detection pipeline 100 also includes a ship type classifier model. In an embodiment, the ship type classifier model uses a MobileNetV3 backbone with a fully connected layer to classify ship detections into a plurality of classes. In an embodiment, the ship classifier model classifies ship detections into the following 8 classes: Tanker, Barge, Cargo Ship, Bulk Carrier, Tugboat, Pleasure Craft, Naval Warship, and Other. In other implementations, other classes may be used (including fewer or additional classes). The pipeline 100 also includes one or more bow direction classifiers. In an embodiment, the bow direction classifiers use a ResNeXt backbone to classify whether the bow of a ship is pointed to the left or the right of the image chips.

[0060] Each component of the pipeline 100 is explained in more detail below.

[0061] The pipeline 100 starts with an image product 102. The image product 102 is an electro-optical satellite image. The image product 102 is received from an image product source. In some embodiments, the image product 102 comes from one of the following image product sources (also referred to as vendors): Maxar, Satellogic, Blacksky, or Sentinel-2. The image products 102 from these sources may have spatialresolutions as high as 0.5m / pix, 0.7m / pix, 1 m / pix, and 10m / pix, respectively, where higher resolution results in better ship detection pipeline performance in general.

[0062] The image product 102 may be delivered in a geotiff format with appropriate pre-processing already applied. Preprocessing may include, for example, atmospheric compensation and orthorectification. In this sense, the image product 102 may be considered analysis ready.

[0063] The image product 102 may be delivered with metadata 106. The image metadata 106 may provide details about image acquisition time, spatial resolution, location footprint, nadir angle, percentage cloud cover, and more.

[0064] In some embodiments, the image product 102 may be an image product 102 from an image or vendor source other than the foregoing. In such cases, the image product 102 may require some basic modifications to adapt to the particular format of packaging the image(s) and metadata files. For example, a first image source (e.g., Maxar) may have such high resolution and large image files, that the image 102 is delivered in multiple tiles with a specified file size. Other image vendor sources may deliver a single image file.

[0065] The pipeline 100 includes an input dropbox 108.

[0066] The ship detection pipeline 100 may be designed as a micro-service with input dropbox 108 configured to constantly check for new image products 102. When a new image product 102 is placed in the input dropbox, the image product 102 is automatically ingested into the system for processing through the pipeline stages described below.

[0067] The image product 102 includes image metadata 106. Before processing a new image product 102, its metadata file 106 is first processed to extract the required information for the ship detection system and final detection report 104.

[0068] A particularly important image metadata property used in image processing is the ground sample distance (GSD). The GSD describes the distance on the ground between the centers of each pixel in the image 102. For a satellite imaging sensor with a specific resolution capability, the best GSD is achieved at the center of an image whenthe satellite has a nadir angle of zero. In practice, the nadir angle is almost never exactly 0 and results in a lower GSD. The GSD is used to select which ship detection model to use in subsequent stages of processing, to best detect the ships in the image 102. Different image product sources, and even the same image product source, may provide images with different GSDs on different occasions. For example, an image product source may provide 0.54 m / pix for a first image on one day, and 0.87 m / pix for a second image on another day. This means that it may not be possible to select the appropriate ship detection model based on the image product source alone. Accordingly, the image GSD may be checked and used to select the most appropriate ship detection model. In the foregoing example, the GSDs would be 0.6 m / pix model for the first image, and the 1.0 m / pix model for the second image.

[0069] In some cases, the GSD may be provided with the image in its metadata file and the GSD is extracted from the metadata file. The image may include geocoordinates of its four comers giving its footprint in the world, which can also be used to compute the resolution of the image in m / pix. This may be done by the system of the present disclosure if, for example, the GSD is missing from the metadata that is included with the image. Generally, the image includes the footprint geocoordinates as such information is used to get the geocoordinates of the detections within the image (as described herein). Accordingly, reference to “extracting a GSD from image metadata” or similar herein may include both obtaining a GSD provided in the metadata file of the satellite image and deriving, by the system, a GSD from other data provided in the image metadata (e.g., four corner geocoordinates).

[0070] Other important metadata 106 such as image acquisition time and image footprint may also be extracted as image metadata 106 from the image product 102 for delivery in the ship reports 104 generated or output by the ship detection pipeline 100. The metadata 106 may be received in a variety of different packaging formats. For example, each image source or provider may provide the image metadata 106 in a different packaging format. Accordingly, the ship detection pipeline 100 may include new image metadata extraction code for extracting image metadata in different formats to support different image product 102 providers or vendors.

[0071] The ship detection pipeline 100 includes an image standardization preprocessing step 110. The preprocessing step 110 may include a merge tiles step.

[0072] Component 110 in the workflow 100 performs additional pre-processing on the electro-optical image products 102 received from the image product sources. The preprocessing includes converting the image product 102 into a more standardized GeoTIFF format for subsequent processing by the machine learning (ML) detection model.

[0073] A “Merge Tiles” step is executed whenever a raw image product 102 is received as multiple tiles. Merging tiles includes stitching the multiple tiles together into a single geotiff image file.

[0074] In addition to merging image tiles (and even if there are no tiles that need to be merged) a further preprocessing step is performed on the image product 102 (or merged image product) to ensure the image has three color channels (e.g., RGB). For panchromatic images with only one channel, the channel is duplicated twice into two additional identical channels (for a total of three channels). For images with additional channels beyond the three standard channels, the additional channels are ignored.

[0075] Once the image standardization is complete, the standardized image is passed on to a subsequent ship detection stage.

[0076] The pipeline 100 includes a detect ships step 112.

[0077] The detect ships step 112 of the ship detection pipeline 100 contains the ship detection models and the modules that streamline the process of running the ship detection models on large geotiffs of arbitrary size.

[0078] The “Detect Ships” (inference) step 112 includes three stages: a preprocessing stage, an inference stage, and a post-processing stage.

[0079] The pre-processing stage includes resampling and chipping the geotiff output from the preprocessing step 110 of the ship detection pipeline 100.

[0080] Spatial resampling includes down-sampling the input image (preprocessed, standardized image product) to the exact spatial resolution that an ML detection model was trained on. There are detection models trained at various resolutions so that change in resolution is minimal.

[0081] As different ML models generally require inputs with a specific shape, large image products are chipped into equal-sized chips. To avoid objects of interest being split across image chips, the image chips are generated with some overlap so that if such an object of interest lies on the edge of one image chip, the object of interest will be entirely contained in the adjacent image chip. Furthermore, this may allow for the image to be processed in parallel using hardware accelerators (i.e., GPUs.)

[0082] The inference stage includes automatically locating ships in the optical satellite imagery. Minimal human intervention is needed. In some cases, no human intervention is needed. In other cases, human intervention may include and be limited to selecting whether a horizontal bounding box model or oriented bounding box model should be used.

[0083] The inference stage includes an inference pipeline. An ML ship detection model may be selected from a list of models trained at different spatial resolutions. The spatial resolution matches the spatial resolution to which the input image product 102 was resampled. In some example embodiments, available image resolutions include 0.6m / pix, 0.7m / pix, 1 m / pix, and 10m / pix. The ML ship detection model may be a deep convolutional neural network model.

[0084] Each image chip obtained from the preprocessing stage of the detect ships step 112 is passed through the deep convolutional neural network model, generating (OBB or HBB) bounding boxes in pixel coordinates, localizing where the ships are predicted to be for each chip. The bounding boxes may be OBB or HBB bounding boxes (e.g., which may be determined by user selection). The bounding boxes may be in pixel coordinates. The image chips may be processed by the neural network model in parallel.

[0085] The post-processing stage includes geocoding, merging chips, and filtering detections.

[0086] Generally, the detection model is configured to accept images of a fixed size (e.g., 1024x1024). While larger images may be chipped into images of this fixed size, they are generated with overlap to avoid splitting a ship in half on the edge of the tiles. This way, if the ship is on the edge in one tile, the ship will be whole in the other. The chip overlap may be set to handle potentially large ships that may be captured in images. Thismay, however, result in smaller ships being wholly in two image chips. Accordingly, the system is configured to filter out duplicates (as described below).

[0087] The pixel coordinates (bounding boxes) of the detection are geocoded into latitude / longitude (lat / lon) coordinates corresponding to their true location in the world. This results in a list of geocoded detections from all the separate image chips that have been merged. The list may contain multiple detections of the same ship.

[0088] The geocoded detections are then filtered by an intersection over area (loA) metric or algorithm. Specifically, the algorithm looks for detections that have large loAs (in geocoded coordinates) and keeps the detection with the largest area. This may avoid redundancy in the detections report.

[0089] An example of filtering using loA will now be described. For loA, the intersection of two polygons can never be more than their area, so loA is between 0 and 1 . Values above 80 or 90% are considered large. When the model makes predictions, it starts with many bounding box candidates covering the image and checks which appear to contain ships, so multiple candidate boxes near a ship can be selected, which can result in multiple detections of the same ship requiring duplicates to be filtered out. Also, to run inference on large images, the images may be chipped up into 1024x1024 pixel chips for processing; however, to prevent a ship from being on the edge and split in two chips, the chips are made with some overlap so a ship on the edge of one image will be fully contained in another image chip. In this case, the model may detect the same ship in both images and one would ideally be filtered out as a duplicate when the results from all the chips are merged back together to produce the final results for the entire image. This process may be imperfect for HBBs as two nearby ships with the right geometry can have significantly overlapping bounding boxes when they are really different ships. This situation may be improved through the use of an OBB model generating OBBs which fit the ships more tightly.

[0090] Once the detections have finished being processed, the detections are saved into a file (“detections file”). The file may be a detections. geojson file. The detections file includes the geocoded detection coordinates of each of the detections. Thedetections file may also include a confidence score for each of the detections that indicates a likelihood the detection is correct.

[0091] The ship detection pipeline 100 includes a ship features determination stage 114.

[0092] The detections file (e.g., detections. geojson file) is passed to the ship features determination stage. The ship features stage 114 chips each detection in the detection list (of the detection file) out of the original image 102 (forming a preview image). The ship features stage 114 also classifies each detection according to their ship type (detected ship type classification). The ship features stage 114 also estimates the detected ship’s length, width and orientation (detected ship characterization).

[0093] The ship features stage 114 includes a preview image generator module (or chip generator module). In creating the preview image (or chip) for a ship detection in the detections file, the preview image generator module creates a PNG image (or other image file type) for each bounding box in the detections file by extracting the bounding box from the original geotiff file, or the standardized image, with a specified amount of padding around the bounding box of the detection. Preview images are saved in a new folder with a supplementary preview images file that indexes all PNG files. The file may be a manifest.json file.

[0094] The ship features stage 114 includes a ship type classification module. The ship type classification module uses a ship classification neural network to process the preview images to predict a ship type from a plurality of ship type classes. In an example embodiment, the ship types include barge, bulk carrier, cargo ship, pleasure craft, naval warship, tanker, tugboat, and other. Prediction scores are produced for all classes on which the model was trained (e.g., eight classes in the foregoing example). The prediction score indicates the likeliness that the ship is of the corresponding class. The predicted class is generally taken to be the class with the highest score.

[0095] The ship features module 114 includes a ship characterization module. The ship characterization module estimates one or more characteristics of the detected ship. Estimated characteristics may include, for example, ship length, ship width, and ship heading. In some cases, which characteristics are estimated by the ship characterizationmodule may be determined based on which ship detection model is used in the detect ships stage 112.

[0096] As noted, the ship characterization module may estimate a ship’s length and width. The detected ship’s dimensions are estimated from the dimensions of the bounding box enclosing the detection. For HBBs, the length may be estimated to be the diagonal of the bounding box which is most accurate when the ship is oriented at 45 degrees, and the width may be estimated to be the shortest dimension of the bounding box. For OBBs, the length and width may be estimated from the long and short dimensions of the bounding box, respectively.

[0097] As noted, the ship characterization module may estimate a detected ship’s heading. In some cases, the heading is only estimated when the OBB detection model is used. The OBB model produces an orientation angle with respect to the long edge of the ship. This orientation may be in radians between -pi and pi, centered on the positive x- axis. This orientation may be converted to degrees with respect to North in geocoordinates. In some cases, a bow direction classifier may be used to predict which direction along the long edge of the ship the bow is located. The orientation angle may be adjusted accordingly, based on the output of the bow direction classifier.

[0098] Once classification and characterization by the ship features module 114 are complete, the results of the ship features module 114 (classification and characterization results) are stored in a new predictions file. The predication file may be a predictions.json file. The predictions file contains the coordinates of each detected ship and the ship type classification and the ship characterization output. For example, the predictions file may contain, for each detected ship, class scores for each of the ship types, and the estimates of length, width, and orientation (for OBB mode only). The predictions file may include the estimated uncertainties for the ship characteristics.

[0099] In some embodiments, the detection models may have built in classifiers and can detect multiple object types. In other embodiments, class labels of ship type may not be available for all ships in the detection dataset (for training the models). In such cases, there may only be ship types for a smaller subset available. In such embodiments, the detector may classify detections as ships and the ship type classifier (e.g., of the shipfeatures module 114) may be trained to predict ship type of a ship detection on a different, smaller dataset.

[0100] The ship detection pipeline 100 includes a ship report generation stage 116.

[0101] Once an image product has finished being processed by the processing pipeline stages 110, 112, 114, the results may be collected in a folder or other data structure. At the end of successful processing, the results folder should contain the detections file (e.g., detections. geojson file) and the image metadata (e.g., image_metadata.json file).

[0102] If there were detected ships in the image 102, there may be an images folder generated and stored containing (i) the image chips of the detected ships (e.g., O.png, l .png, 2. png ... ), (ii) a preview images file (e.g., manifest.json file) that lists the images in the images folder, and (iii) a predictions file (e.g., predictions.json file) that lists all the detection coordinates, the corresponding image chip file name, the ship type class scores, and the additional characterization properties (e.g., length, width, orientation, and the standard deviation representing their uncertainties). The results folder may automatically be saved in an archive storage 118 in an archive file format that supports data compression (e.g., zip file).

[0103] The ship detection pipeline 100 includes an output dropbox 120.

[0104] The output dropbox is a folder where the ship reports are automatically stored after processing is complete. This output folder 120 may be configured to be the ingest folder for a particular maritime surveillance software application instance (to which the ship detection pipeline 100 interfaces and provides reports to be displayed or visualized) where the results should be ingested and displayed. This may allow for a human operator to check the results before ingestion into the maritime surveillance software application, for purposes of quality assurance.

[0105] The ship detection pipeline 100 generates a report 104.

[0106] The report 104 may be an individual archive folder (e.g., zipped folder) that is produced by the ship report generation stage 116 and output to the output dropbox 120, as described above.

[0107] In an embodiment, the report 104 includes at least the geocoded coordinates of the detection bounding box (e.g., latitude / longitude rather than pixel row and columns) for each detection in the ship detection output. The geocoded coordinates inform where in the world the ship is located rather than simply where in the image. The report 104 may also include a confidence score for each detection.

[0108] The report 104 may include the ship classification output or the ship characterization output (or some subsets thereof). Length, width, and orientation may be most useful for identifying which ship is actually in the image. Ship type information, on the other hand, may be beneficial but may not be as useful for identifying the actual ship or helping to assess its direction of travel and track association.

[0109] Referring now to Figure 2, shown therein is a computer system 200 for automated ship detection in optical satellite imagery, according to an embodiment. The system 200 may be configured to implement the pipeline 100 of Figure 1 .

[0110] The system 200 may be implemented by the system 10 of Figure 11 , described below.

[0111] Components of the computer system 200 may be implemented at one or more devices, such as a server platform (e.g., server platform 12 of Figures 11 ) and a user device (e.g., user devices 18 of Figure 11 ).

[0112] The system 200 includes a memory 202 and a processor 204 in communication with the memory 202. The processor 204 and memory 202 may be located at the server 12 of Figure 11 . The processor 204 is configured to execute various software modules and components. In some embodiments, modules or components executed by the processor 204 may include server-side software components and clientside software components that communicate with each other in order to provide various features and functionalities of the system 200. In some cases, server-side components may be executed at server 12 and client-side components may be executed at user device 18 of Figure 11 .

[0113] The system 200 includes a communication interface device 206 for transmitting and receiving data to and from other computing devices. The communicationinterface device 206 may include a network interface device for transmitting and receiving data via a network connection (e.g., local area network, wide area network, etc.).

[0114] The system 200 includes a display device 208 for displaying data generated by the system 200. The display device 208 may be located at the user device 18 of Figure 11.

[0115] The system 200 includes an input device 210 for receiving input data from a user interacting with the system 200. For example, a user may use input device 210 to interact with the system 200 through a graphical user interface generated by the processor 204 and displayed via the display device 208. The input device 210 may be located at the user device 18.

[0116] The system 206 is in communication with an optical satellite image product source 212. The system 206 receives an electro-optical satellite image 214, which is to be processed by the system 200, from the image product source 212 via the communication interface 206. The satellite image 214 is stored in memory 202.

[0117] The processor 204 is configured to execute a plurality of software modules or components including an image product collector module 216, an automatic image product ingestion service 218, an image metadata extractor module 220, an image product preprocessing module 222, a ship detection module 224, a ship features determination module 226, a ship report generator module 228, a ship report storage module 230, and a graphical user interface module 232.

[0118] The image product collector module 216 is configured to collect the optical satellite image 214 from the image product source 212. The image product collector module 216 includes an automatic image product ingestion service 218. The automatic image product ingestion service 218 may operate similarly to the input dropbox 108 of Figure 1 .

[0119] The image metadata extractor module 220 is configured to receive the optical satellite image 214 as input and extract image metadata 234 from the optical satellite image 214. The image metadata 234 is stored in memory 202. The image metadata 234 may be the image metadata 106 of Figure 1 .

[0120] In an embodiment, as shown in Figure 3, the image metadata 234 includes a ground sample distance 302, an image acquisition time 304, and an image footprint 306.

[0121] The image product preprocessing module 222 receives the optical satellite image 214 as input and converts the image 214 into a standardized format for processing by one or more machine learning detection models. The standardized format may be a geotiff format. Accordingly, the image product preprocessing module 222 outputs a standardized image 236. The standardized image 236 is stored in memory 202. The image product preprocessing module 222 may operate similarly to the image standardization preprocessing step 110 of Figure 1.

[0122] The ship detection module 224 is configured to receive the standardized image 236 as input and detect ships in the standardized image 236. The ship detection module 224 outputs a detections file 238. The detections file is stored in memory 202. The ship detection module 224 may operate similarly to the detect ships stage 112 of Figure 1 .

[0123] In an embodiment, as shown in Figure 4, the ship detection module 224 includes a detection preprocessing module 402, a detection inference module 404, and a detection post-processing module 406. The detection preprocessing module 402, the detection inference module 404, and the detection post-processing module 406 may operate similarly to the pre-processing, inference, and post-processing stages of the detect ships stage 112 of Figure 1 , respectively.

[0124] The detection pre-processing module 402 receives the standardized image 236 as input and generates pre-processed image data 240 as output. This may include spatially resampling and chipping the standardized image 236. Accordingly, the preprocessed image data 240 may include a plurality of image chips generated from the standardized image 236. The pre-processed image data 240 is stored in memory 202.

[0125] In an embodiment, as shown in Figure 5, the detection inference module 404 includes a horizontal bounding box model 502 and an oriented bounding box model 504. In some embodiments, the detection inference module 404 includes a plurality ofHBB models 502 and a plurality of OBB models 504. In other embodiments, the detection inference model 404 may include only HBB models or only OBB models.

[0126] Whether the HBB model 502 or the OBB model 504 is used may be determined automatically by the ship detection module 224 or by a user from a list of available models. User selection of the model may be provided via the graphical user interface module 232 as model type selection data 242.

[0127] The detection inference module 404 receives the preprocessed image data 240 as input and generates inference data 244 as output. As shown in Figure 6, the inference data 244 includes detected ship localization data 602. The detected ship localization data 603 includes, for each chip generated by the detection preprocessing module 402, a bounding box in pixel coordinates for each object (ship) detected in the image chip.

[0128] The detection post-processing module 406 receives the inference data 244 as input and outputs post-processed image data 246. The post-processed image data 246 is stored in memory 202.

[0129] In an embodiment, as shown in Figure 7, the post-processed image data 246 includes geocoded detections 702, merged image chips 704, and filtered geocoded detections 706.

[0130] The detections file 238 generated by the ship detection module 224 includes a list of detections, where each detection has associated geocoded coordinates (determined by the post-processing module 406) and confidence score.

[0131] For example, as illustrated in Figure 8, according to an embodiment, the detections file 238 may include detections 802-1 , 802-2, 802-n. Detection 802-1 includes detection coordinates 804-1 and detection confidence score 806-1. Detection 802-2 includes detection coordinates 804-2 and detection confidence score 806-2. Detection 802-n includes detection coordinates 804-n and detection confidence score 806-n.

[0132] Referring again to Figure 2, the ship features determination module 226 receives the detections file 238 as input and generates a predictions file 248. Thepredictions file 248 is stored in memory 202. The ship features determination module 226 may operate similarly to the ship features stage 114 of pipeline 100 of Figure 1 .

[0133] In an embodiment, as illustrated in Figure 9, the ship features determination module 226 includes a preview image generator module 902, a detected ship classification module 904, and a detected ship characterization module 906.

[0134] The preview image generator module 902 generates a chipped preview image 250 (e.g., PNG image) for each bounding box 804 in the detections file 238 by extracting the bounding box from the original geotiff file or from the standardized image. The extraction may include a specified amount of padding around the bounding box of the detection. In an embodiment, the preview image 250 is generated from the standardized image 236.

[0135] The preview image generator module 902 also generates a preview images file 252 that indexes all preview image files 250 generated by the preview image generator module 902.

[0136] The detected ship type classification module 904 includes a neural network that receives the preview images 250 as input and processes the images 250 to predict a ship type. The classification module 904 generates a class label that is assigned to the preview image 250. The ship type classification module 904 analyzes a preview image 250 via the neural network model and generates detected ship classification data 254. The detected ship classification data 254 is stored in the memory 202. The detected ship classification data 254 includes a prediction score for each class on which the classification model was trained. The prediction score indicates a likelihood that the detected ship is of the corresponding class. The detected ship type classification module 904 may then select the class with the highest prediction score and assign the corresponding class to the detected ship.

[0137] The detected ship characterization module 906 receives a preview image 250 as input and processes the preview image 250 to obtain ship feature characterization data 256. The ship feature characterization data 256 is stored in memory 202.

[0138] The detected ship characterization module 906 is configured to estimate length and width of the detected ship using the dimensions of the bounding box enclosing the detection in the preview image 250. The length and width are stored as ship feature characterization data 256.

[0139] In cases where the OBB detection model 504 of the detection inference module 404 is used, the detected ship characterization module 906 is further configured to estimate a heading or orientation of the detected ship. The heading or orientation of the detection ship is stored as ship feature characterization data 256.

[0140] The ship features determination module 226 stores the ship classification data 254 and the ship characterization data 256 in the predictions file 248.

[0141] The predictions file 248 contains coordinates of each detected ship, class scores for each of the ship types, and estimates of length, width, and orientation (if OBB model used). The predictions file 248 may also include estimated uncertainties for the ship characteristics.

[0142] An example predictions file 248 is shown in Figure 10. The predictions file 248 includes detections 1002-1 , 1002-2, and 1002-n. Each of the detections 1002 includes detection coordinates 1004, a detected ship type classification 1006, a detected ship characterization output 1008 (e.g., including length, width, and, if OBB mode is used, an orientation), and an image chip file name 1010.

[0143] Referring again to Figure 2, the ship report generator module 228 generates a detection report 258. The detection report 258 is stored in memory 202. The detection report 258 may be the report 104 of pipeline 100 of Figure 1. The detection report 258 includes data from the detections file 238, image metadata 234, preview images index file 252, and predictions file 248. The ship detection report 258 may be saved in a compressed format. The detection report 258 includes at least the geocoded coordinates of the bounding box for each ship detection in the ship detection output.

[0144] The graphical user interface module 232 is configured to display the ship detection report 258, or some subset of data contained therein, in a graphical user interface displayed at the display device 208.

[0145] Referring now to Figure 11 , shown therein is a system 10 for automated ship detection, according to an embodiment.

[0146] The system 10 includes a ship detection server platform 12 which communicates with a plurality of satellite electro-optical image source devices 14, an earth observation software application server platform 16, and a plurality of user devices 18 via a network 20. The server platform 12 may be a purpose-built machine designed specifically for performing automated ship detection on optical satellite images and generating ship reports.

[0147] The server platform 12, image source devices 14, server platform 16, and user devices 18 may be a server computer, desktop computer, notebook computer, tablet, PDA, smartphone, or another computing device. The devices 12, 14, 16, 18 may include a connection with the network 20 such as a wired or wireless connection to the Internet. In some cases, the network 20 may include other types of computer or telecommunication networks. The devices 12, 14, 16, 18 may include one or more of a memory, a secondary storage device, a processor, an input device, a display device, and an output device. Memory may include random access memory (RAM) or similar types of memory. Also, memory may store one or more applications for execution by processor. Applications may correspond with software modules comprising computer executable instructions to perform processing for the functions described below. Secondary storage device may include a hard disk drive, floppy disk drive, CD drive, DVD drive, Blu-ray drive, or other types of non-volatile data storage. Processor may execute applications, computer readable instructions or programs. The applications, computer readable instructions or programs may be stored in memory or in secondary storage or may be received from the Internet or other network 20. Input device may include any device for entering information into device 12, 14, 16, 18. For example, input device may be a keyboard, keypad, cursorcontrol device, touchscreen, camera, or microphone. Display device may include any type of device for presenting visual information. For example, display device may be a computer monitor, a flat-screen display, a projector or a display panel. Output device may include any type of device for presenting a hard copy of information, such as a printer for example. Output device may also include other types of output devices such as speakers, for example. In some cases, device 12, 14, 16, 18 may include multiple of any one ormore of processors, applications, software modules, second storage devices, network connections, input devices, output devices, and display devices.

[0148] Although devices 12, 14, 16, 18 are described with various components, one skilled in the art will appreciate that the devices 12, 14, 16, 18 may in some cases contain fewer, additional or different components. In addition, although aspects of an implementation of the devices 12, 14, 16, 18 may be described as being stored in memory, one skilled in the art will appreciate that these aspects can also be stored on or read from other types of computer program products or computer-readable media, such as secondary storage devices, including hard disks, floppy disks, CDs, or DVDs; a carrier wave from the Internet or other network; or other forms of RAM or ROM. The computer- readable media may include instructions for controlling the devices 12, 14, 16, 18 and / or processor to perform a particular method.

[0149] In the description that follows, devices such as server platform 12, image source devices 14, server platform 16, and user devices 18 are described performing certain acts. It will be appreciated that any one or more of these devices may perform an act automatically or in response to an interaction by a user of that device. That is, the user of the device may manipulate one or more input devices (e.g., a touchscreen, a mouse, or a button) causing the device to perform the described act. In many cases, this aspect may not be described below, but it will be understood.

[0150] As an example, it is described below that the devices 12, 14, 16, 18 may send information to the server platform 12. For example, a user using the user device 18 may manipulate one or more input devices (e.g., a mouse and a keyboard) to interact with a user interface displayed on a display of the user device 18. Generally, the device may receive a user interface from the network 20 (e.g., in the form of a webpage). Alternatively, or in addition, a user interface may be stored locally at a device (e.g., a cache of a webpage or a mobile application).

[0151] Server platform 12 may be configured to receive a plurality of information, from each of the plurality of image source devices 14, server platform 16, and user devices 18. Generally, the information may comprise at least an identifier identifying thedevice or user. For example, the information may comprise one or more of a username, e-mail address, password, device identifier, or social media handle.

[0152] In response to receiving information, the server platform 12 may store the information in storage database. The storage may correspond with secondary storage of the device 12, 14, 16, 18. Generally, the storage database may be any suitable storage device such as a hard disk drive, a solid state drive, a memory card, or a disk (e.g., CD, DVD, or Blu-ray etc.). Also, the storage database may be locally connected with server platform 12. In some cases, storage database may be located remotely from server platform 12 and accessible to server platform 12 across a network for example. In some cases, storage database may comprise one or more storage devices located at a networked cloud storage provider.

[0153] The image source device 14 may be associated with an image source account. Similarly, the user device 18 may be associated with a user account. Any suitable mechanism for associating a device with an account is expressly contemplated. In some cases, a device may be associated with an account by sending credentials (e.g., a cookie, login, or password etc.) to the server platform 12. The server platform 12 may verify the credentials (e.g., determine that the received password matches a password associated with the account). If a device is associated with an account, the server platform 12 may consider further acts by that device to be associated with that account.

[0154] Referring now to Figure 2, shown therein is a block diagram of a computing device 1000 of the system 10 of Figure 1 , according to an embodiment. The computing device 1000 may be, for example, any one of devices 12, 14, 16, 18 of Figure 11 .

[0155] The computing device 1000 includes multiple components such as a processor 1020 that controls the operations of the computing device 1000. Communication functions, including data communications, voice communications, or both may be performed through a communication subsystem 1040. Data received by the computing device 1000 may be decompressed and decrypted by a decoder 1060. The communication subsystem 1040 may receive messages from and send messages to a wireless network 1500.

[0156] The wireless network 1500 may be any type of wireless network, including, but not limited to, data-centric wireless networks, voice-centric wireless networks, and dual-mode networks that support both voice and data communications.

[0157] The computing device 1000 may be a battery-powered device and as shown includes a battery interface 1420 for receiving one or more rechargeable batteries 1440.

[0158] The processor 1020 also interacts with additional subsystems such as a Random Access Memory (RAM) 1080, a flash memory 1100, a display 1120 (e.g. with a touch-sensitive overlay 1140 connected to an electronic controller 1160 that together comprise a touch-sensitive display 1180), an actuator assembly 1200, one or more optional force sensors 1220, an auxiliary input / output (I / O) subsystem 1240, a data port 1260, a speaker 1280, a microphone 1300, short-range communications systems 1320 and other device subsystems 1340.

[0159] In some embodiments, user-interaction with the graphical user interface may be performed through the touch-sensitive overlay 1140. The processor 1020 may interact with the touch-sensitive overlay 1140 via the electronic controller 1160. Information, such as text, characters, symbols, images, icons, and other items that may be displayed or rendered on a computing device generated by the processor 102 may be displayed on the touch-sensitive display 118.

[0160] The processor 1020 may also interact with an accelerometer 1360 as shown in Figure 12. The accelerometer 1360 may be utilized for detecting direction of gravitational forces or gravity-induced reaction forces.

[0161] To identify a subscriber for network access according to the present embodiment, the computing device 1000 may use a Subscriber Identity Module or a Removable User Identity Module (SIM / RUIM) card 1380 inserted into a SIM / RUIM interface 1400 for communication with a network (such as the wireless network 1500). Alternatively, user identification information may be programmed into the flash memory 1100 or performed using other techniques.

[0162] The computing device 1000 also includes an operating system 1460 and software components 1480 that are executed by the processor 1020 and which may be stored in a persistent data storage device such as the flash memory 1100. Additional applications may be loaded onto the computing device 1000 through the wireless network 1500, the auxiliary I / O subsystem 1240, the data port 1260, the short-range communications subsystem 1320, or any other suitable device subsystem 1340.

[0163] In use, a received signal such as a text message, an e-mail message, web page download, or other data may be processed by the communication subsystem 1040 and input to the processor 1020. The processor 1020 then processes the received signal for output to the display 1120 or alternatively to the auxiliary I / O subsystem 1240. A subscriber may also compose data items, such as e-mail messages, for example, which may be transmitted over the wireless network 1500 through the communication subsystem 1040.

[0164] For voice communications, the overall operation of the computing device 1000 may be similar. The speaker 1280 may output audible information converted from electrical signals, and the microphone 1300 may convert audible information into

[0165] While the above description provides examples of one or more apparatus, methods, or systems, it will be appreciated that other apparatus, methods, or systems may be within the scope of the claims as interpreted by one of skill in the art.

Claims

Claims:

1. A method of automatically locating and characterizing ships in electro-optical satellite imagery, the method comprising: storing, in a data storage device, an electro-optical satellite image comprising image metadata; extracting, using at least one processor, a ground sample distance from the image metadata; selecting, using the at least one processor, a ship detection model from a plurality of ship detection models each configured to detect ships in the electro-optical satellite image and localize each detected ship using a bounding box defined by bounding box coordinates, wherein the ship detection model is selected using the extracted ground sample distance; converting, using the at least one processor, the electro-optical satellite image into a standardized format (“standardized image”) for subsequent processing by the ship detection model; processing, using the at least one processor, the standardized image using the selected ship detection model to obtain a ship detection output in which each ship detection detected by the ship detection model is defined by a bounding box and an associated confidence score; for each ship detection in the ship detection output, geocoding, using the at least one processor, pixel coordinates of the bounding box into geocoded coordinates comprising latitude and longitude coordinates; generating, using the at least one processor, a ship detection report including the geocoded coordinates of each ship detection in the ship detection output; andtransmitting, via a communication interface, the ship detection report to a user computing device configured to display the ship detection report in a graphical user interface.

2. The method of claim 1 , further comprising: for each ship detection in the ship detection output, extracting, using the at least one processor, a preview image comprising the corresponding bounding box enclosing the detection from the standardized image; for each preview image, processing the preview image using a ship characterization module to obtain a ship characterization output including a length and width of the ship detection, the ship characterization module configured to estimate the length and width from the dimensions of the bounding box enclosing the ship detection; wherein the ship detection report further includes the length and width of each ship detection in the ship detection output.

3. The method of claim 2, wherein the selected ship detection model is an oriented bounding box model, wherein the ship characterization module is further configured to estimate an orientation of the ship detection, and wherein the ship detection report further includes the orientation of each ship detection in the ship detection output.

4. The method of claim 2, further comprising: for each preview image, processing, using the at least one processor, the preview image using a ship type classification model to obtain a ship classification output including a predicted ship type for the ship detection; andwherein the ship detection report further includes the predicted ship type for each ship detection in the ship detection output.

5. The method of claim 1 , wherein the electro-optical satellite image is in a GeoTIFF format with atmospheric compensation and orthorectification already applied.

6. The method of claim 1 , wherein the ground sample distance describes a distance on ground between centers of each pixel.

7. The method of claim 1 , wherein converting the electro-optical satellite image comprises processing the electro-optical satellite image to have only red, green, and blue colour channels.

8. The method of claim 7, wherein converting the electro-optical satellite image further comprises: where the electro-optical satellite image is panchromatic with only one channel, duplicating the one channel twice to obtain three identical channels, the three identical channels being the RGB channels; and where the electro-optical satellite image includes additional channels beyond the red, green, and blue channels, ignoring the additional channels.

9. The method of claim 1 , wherein the electro-optical satellite image comprises a plurality of tiles, and wherein converting the electro-optical satellite image comprises stitching the plurality of tiles into a single geotiff image file.

10. The method of claim 1 , further comprising performing, using the at least one processor, spatial resampling on the standardized image to the same spatial resolution that the selected ship detection model was trained on.

11. The method of claim 1 , further comprising generating, using the at least one processor, a plurality of equally sized image chips from the standardized image, wherein the image chips are generated with some overlap, and wherein processing the standardized image using the selected ship detection model comprises processing the plurality of equally sized image chips using the selected ship detection model.

12. The method of claim 1 , wherein the plurality of ship detection models include at least one horizontal bounding box (“HBB”) model and at least one oriented bounding box (“OBB”) model.

13. The method of claim 1 , wherein the plurality of ship detection models include at least one oriented bounding box model.

14. The method of claim 1 , wherein the plurality of ship detection models have been trained at different spatial resolutions, and wherein the selected ship detection model and the standardized image have the same spatial resolution, after the standardized image has been resampled to that spatial resolution.

15. The method of claim 1 , further comprising, for each ship detection in the ship detection output, geocoding, using the at least one processor, pixel coordinates of the bounding box into a geocoded detection comprising latitude and longitude coordinates, and wherein the latitude and longitude coordinates are included in the ship detection output.

16. The method of claim 15, further comprising filtering, using the at least one processor, the geocoded detections using an intersection over area (“loA”) metric to identify overlapping detections, determining that the overlapping detections have an loA meet an loA size threshold, and retaining only one of the overlapping detections with the largest area.

7. A computer system for automatically locating and characterizing ships in electro- optical satellite imagery, the system comprising: a communication interface device for: receiving an electro-optical satellite image comprising image metadata; and transmitting a ship detection report to a user computing device configured to display the ship detection report in a graphical user interface; a data storage device for storing: the electro-optical satellite image; and a plurality of ship detection machine learning models each configured to detect ships in the electro-optical satellite image and localize each detected ship using a bounding box defined by bounding box coordinates; at least one processor configured to: extract a ground sample distance from the image metadata; select a ship detection model from the plurality of ship detection models, wherein the ship detection model is selected using the extracted ground sample distance; convert the electro-optical satellite image into a standardized format for subsequent processing by the ship detection model to obtain a standardized image; process the standardized image using the selected ship detection model to obtain a ship detection output in which each ship detection detected by theship detection model is defined by a bounding box and an associated confidence score; for each ship detection in the ship detection output, extracting a preview image comprising the corresponding bounding box enclosing the detection from the standardized image; for each preview image, processing the preview image using a ship type classification model to obtain a ship classification output including a predicted ship type for the ship detection; for each preview image, processing the preview image using a ship characterization module to obtain a ship characterization output including a length and width of the ship detection, the ship characterization module configured to estimate the length and width from the dimensions of the bounding box enclosing the ship detection; generating the ship detection report, the ship report including (i) the ship detection output and (ii) the ship classification and ship characterization outputs for each ship detection in the ship detection output.

18. A non-transitory computer readable storage medium storing processor-executable instructions which, when executed by at least one processor, cause the at least one processor to perform a method of automatically locating and characterizing ships in electro-optical satellite imagery, the method comprising: storing, in a data storage device, an electro-optical satellite image comprising image metadata; extracting, using at least one processor, a ground sample distance from the image metadata;selecting, using the at least one processor, a ship detection model from a plurality of ship detection models each configured to detect ships in the electro-optical satellite image and localize each detected ship using a bounding box defined by bounding box coordinates, wherein the ship detection model is selected using the extracted ground sample distance; converting, using the at least one processor, the electro-optical satellite image into a standardized format (“standardized image”) for subsequent processing by the ship detection model; processing, using the at least one processor, the standardized image using the selected ship detection model to obtain a ship detection output in which each ship detection detected by the ship detection model is defined by a bounding box and an associated confidence score; for each ship detection in the ship detection output, extracting, using the at least one processor, a preview image comprising the corresponding bounding box enclosing the detection from the standardized image; for each preview image, processing, using the at least one processor, the preview image using a ship type classification model to obtain a ship classification output including a predicted ship type for the ship detection; for each preview image, processing the preview image using a ship characterization module to obtain a ship characterization output including a length and width of the ship detection, the ship characterization module configured to estimate the length and width from the dimensions of the bounding box enclosing the ship detection; generating, using the at least one processor, a ship detection report including (i) the ship detection output and (ii) the ship classification and ship characterization outputs for each ship detection in the ship detection output; andtransmitting, via a communication interface, the ship detection report to a user computing device configured to display the ship detection report in a graphical user interface.

19. A computer system comprising at least one processor configured to execute the method of any one of claims 1 -16.

20. A non-transitory computer readable medium storing processor-executable instructions which, when executed by at least one processor, cause the processor to perform the method of any one of claims 1 -16.

Images