System and method for determining vessel direction of travel in synthetic aperture radar imagery

Abstract

A system and method for determining vessel heading in synthetic aperture radar (SAR) imagery is provided. The method includes: storing, in a data storage device: a vessel signature comprising a set of SAR image pixels corresponding to a vessel detected in a SAR image containing the set of SAR image pixels; and an orientation of the vessel, the orientation having a 180 degree ambiguity; processing the vessel signature with a vessel direction determination module configured to determine a direction of travel of the vessel relative to a SAR payload that captured the SAR image based on azimuth smearing; and determining a heading of the vessel by disambiguating the orientation using the determined direction of travel.

Description

SYSTEM AND METHOD FOR DETERMINING VESSEL DIRECTION OF TRAVEL IN SYNTHETIC APERTURE RADAR IMAGERYTechnical Field

[0001] The following relates generally to maritime surveillance using synthetic aperture radar (SAR), and more particularly to systems and methods for determining vessel direction of travel in SAR imagery.Introduction

[0002] Knowing which way or direction a vessel is travelling is very important to users engaged in wide-area maritime surveillance. For example, a particular goal in SAR vessel detection is to be able to say that a ship is traveling in a particular heading, relative to North. For example, a heading of 45 degrees means the ship is heading north-east.

[0003] Vessel orientation can be estimated from SAR imagery. The vessel orientation estimation presents a 180 degree ambiguity problem, where there are two possible directions of travel (e.g., two angles relative to north, separated by 180 degrees). That is, an angle of the vessel relative to north can be determined, but it is not known which end is the bow and which is the stern. To determine the heading of the vessel, the vessel orientation must be disambiguated.

[0004] Accordingly, there is a need for an improved system and method for determining vessel direction of travel in SAR imagery that overcomes at least some of the disadvantages of existing systems and methods.Summary

[0005] A system for determining vessel heading in synthetic aperture radar (SAR) imagery is provided. The system includes a computer memory that stores: a vessel signature comprising a set of SAR image pixels corresponding to a vessel detected in a SAR image containing the set of SAR image pixels; and an orientation of the vessel, the orientation having a 180 degree ambiguity. The system also includes one or more processors in communication with the computer memory and configured to: process the vessel signature with a vessel direction determination module configured to determine a direction of travel of the vessel relative to a SAR payload that captured the SAR imagebased on azimuth smearing; and determine a heading of the vessel by disambiguating the orientation using the determined direction of travel.

[0006] In an embodiment, the vessel direction determination module executes an algorithm that exploits azimuth smearing in the SAR image to determine the direction of travel.

[0007] In an embodiment, the algorithm: identifies pixel regions in the SAR image relative to the vessel signature where azimuth smearing could occur, wherein the identifying includes excluding from the pixel regions a set of actual vessel pixels; determines that azimuth smearing is present in at least one of the pixel regions; and determines the direction of travel of the vessel based on a location of the at least one pixel region with azimuth smearing relative to the vessel.

[0008] In an embodiment, the algorithm excludes the set of actual vessel pixels by excluding pixels within a shaped defined using an estimated length and width of the vessel and the orientation of the vessel.

[0009] In an embodiment, identifying the pixel regions includes excluding pixel regions in front of the vessel and behind the vessel.

[0010] In an embodiment, the algorithm determines that azimuth smearing is present by comparing the pixel regions where azimuth smearing could occur to the excluded pixel regions.

[0011] In an embodiment, the algorithm determines that azimuth smearing is present by comparing pixel values of the pixel regions where azimuth smearing could occur to a smearing threshold and identifies azimuth smearing in the pixel regions when the pixel values exceed the smearing threshold.

[0012] In an embodiment, the smearing threshold is set lower than a threshold used to identify the vessel signature.

[0013] In an embodiment, the pixel values are pixel brightness values.

[0014] In an embodiment, the algorithm determines that azimuth smearing by determining whether the pixel regions are statistically different from the excluded pixel regions.

[0015] In an embodiment, the algorithm: identifies pixel regions in the SAR image relative to the vessel signature where azimuth smearing could occur, where those pixel regions exclude a set of actual vessel pixels; and determines the direction of travel of the vessel based on which of the pixel regions contains azimuth smearing.

[0016] In an embodiment, the algorithm is a machine learning model.

[0017] In an embodiment, the machine learning receives an image chip containing the vessel signature as input and returns a value indicating the direction of travel of the vessel.

[0018] In an embodiment, the direction of vessel travel is away from the SAR payload or towards the SAR payload.

[0019] In an embodiment, the machine learning model is trained using a method comprising: providing a set of SAR image chips containing vessel signatures, the set including SAR image chips with azimuth smearing and SAR image chips without azimuth smearing; determining, for each vessel signature, whether the vessel was headed towards or away from the SAR payload using a known heading of the vessel; labelling the SAR image chips in the set by appending a label indicating whether the vessel in the SAR image chip was heading towards or away from the SAR payload; and training the machine learning model using the labelled set of SAR image chips.

[0020] A method of determining vessel heading in synthetic aperture radar (SAR) imagery is also provided. The method includes: storing, in a data storage device: a vessel signature comprising a set of SAR image pixels corresponding to a vessel detected in a SAR image containing the set of SAR image pixels; and an orientation of the vessel, the orientation having a 180 degree ambiguity; processing the vessel signature with a vessel direction determination module configured to determine a direction of travel of the vessel relative to a SAR payload that captured the SAR image based on azimuth smearing; and determining a heading of the vessel by disambiguating the orientation using the determined direction of travel.

[0021] In an embodiment, the vessel direction determination module executes an algorithm that exploits azimuth smearing in the SAR image to determine the direction of travel.

[0022] In an embodiment, the algorithm: identifies pixel regions in the SAR image relative to the vessel signature where azimuth smearing could occur, wherein the identifying includes excluding from the pixel regions a set of actual vessel pixels; determines that azimuth smearing is present in at least one of the pixel regions; and determines the direction of travel of the vessel based on a location of the at least one pixel region with azimuth smearing relative to the vessel.

[0023] In an embodiment, the algorithm excludes the set of actual vessel pixels by excluding pixels within a shaped defined using an estimated length and width of the vessel and the orientation of the vessel.

[0024] In an embodiment, identifying the pixel regions includes excluding pixel regions in front of the vessel and behind the vessel.

[0025] In an embodiment, the algorithm determines that azimuth smearing is present by comparing the pixel regions where azimuth smearing could occur to the excluded pixel regions.

[0026] In an embodiment, the algorithm determines that azimuth smearing is present by comparing pixel values of the pixel regions where azimuth smearing could occur to a smearing threshold and identifies azimuth smearing in the pixel regions when the pixel values exceed the smearing threshold.

[0027] In an embodiment, the smearing threshold is set lower than a threshold used to identify the vessel signature.

[0028] In an embodiment, the pixel values are pixel brightness values.

[0029] In an embodiment, the algorithm determines that azimuth smearing is present by determining whether the pixel regions are statistically different from the excluded pixel regions.

[0030] In an embodiment, the algorithm: identifies pixel regions in the SAR image relative to the vessel signature where azimuth smearing could occur, where those pixelregions exclude a set of actual vessel pixels; and determines the direction of travel of the vessel based on which of the pixel regions contains azimuth smearing.

[0031] In an embodiment, the algorithm is a machine learning model.

[0032] In an embodiment, the machine learning receives an image chip containing the vessel signature as input and returns a value indicating the direction of travel of the vessel.

[0033] In an embodiment, the direction of vessel travel is away from the SAR payload or towards the SAR payload.

[0034] In an embodiment, the machine learning model is trained using a method comprising: providing a set of SAR image chips containing vessel signatures, the set including SAR image chips with azimuth smearing and SAR image chips without azimuth smearing; determining, for each vessel signature, whether the vessel was headed towards or away from the SAR payload using a known heading of the vessel; labelling the SAR image chips in the set by appending a label indicating whether the vessel in the SAR image chip was heading towards or away from the SAR payload; and training the machine learning model using the labelled set of SAR image chips.

[0035] A method of determining vessel direction of travel in SAR imagery using azimuth smearing is provided. The method includes: providing a set of SAR image chips containing vessel signatures, the set including SAR image chips with azimuth smearing and SAR image chips without azimuth smearing; determining, for each vessel signature, whether the vessel was headed towards or away from the SAR payload using a known heading of the vessel; labelling the SAR image chips in the set by appending a label indicating whether the vessel in the SAR image chip was heading towards or away from the SAR payload; training the machine learning model using the labelled set of SAR image chips; and using the trained machine learning model to determine a direction of travel of a vessel detected in a SAR image.

[0036] Other aspects and features will become apparent, to those ordinarily skilled in the art, upon review of the following description of some exemplary embodiments.Brief Description of the Drawings

[0037] The drawings included herewith are for illustrating various examples of articles, methods, and apparatuses of the present specification. In the drawings:

[0038] Figure 1 is a schematic diagram of a system for vessel target detection and characterization in SAR imagery, according to an embodiment;

[0039] Figure 2 is a block diagram of a computer system for determining vessel heading, according to an embodiment;

[0040] Figure 3 is a flowchart of a method of training a machine learning for determining vessel direction of travel in SAR imagery based on azimuth smearing, according to an embodiment;

[0041] Figure 4 is a SAR satellite image with four ships with superstructure that may be learned by a machine learning model for determining vessel direction of travel, according to an embodiment;

[0042] Figure 5 is a SAR image chip including a vessel signature, according to an embodiment; and

[0043] Figure 6 is the SAR image chip of Figure 5 with multiple regions superimposed and identified, the regions for use in determining vessel direction of travel, according to an embodiment.Detailed Description

[0044] Various apparatuses or processes will be described below to provide an example of each claimed embodiment. No embodiment described below limits any claimed embodiment and any claimed embodiment may cover processes or apparatuses that differ from those described below. The claimed embodiments are not limited to apparatuses or processes having all of the features of any one apparatus or process described below or to features common to multiple or all of the apparatuses described below.

[0045] One or more systems described herein may be implemented in computer programs executing on programmable computers, each comprising at least oneprocessor, a data storage system (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device. For example, and without limitation, the programmable computer may be a programmable logic unit, a mainframe computer, server, and personal computer, cloud-based program or system, laptop, personal data assistance, cellular telephone, smartphone, or tablet device.

[0046] Each program is preferably implemented in a high-level procedural or object-oriented programming and / or scripting language to communicate with a computer system. However, the programs can be implemented in assembly or machine language, if desired. In any case, the language may be a compiled or interpreted language. Each such computer program is preferably stored on a storage media or a device readable by a general or special purpose programmable computer for configuring and operating the computer when the storage media or device is read by the computer to perform the procedures described herein.

[0047] A description of an embodiment with several components in communication with each other does not imply that all such components are required. On the contrary, a variety of optional components are described to illustrate the wide variety of possible embodiments of the present invention.

[0048] Further, although process steps, method steps, algorithms or the like may be described (in the disclosure and I or in the claims) in a sequential order, such processes, methods and algorithms may be configured to work in alternate orders. In other words, any sequence or order of steps that may be described does not necessarily indicate a requirement that the steps be performed in that order. The steps of processes described herein may be performed in any order that is practical. Further, some steps may be performed simultaneously.

[0049] When a single device or article is described herein, it will be readily apparent that more than one device I article (whether or not they cooperate) may be used in place of a single device I article. Similarly, where more than one device or article is described herein (whether or not they cooperate), it will be readily apparent that a single device I article may be used in place of the more than one device or article.

[0050] The present disclosure provides systems and methods that exploit azimuth smearing in SAR imagery of detected vessel to determine a heading of the vessel. In particular, azimuth smearing may be exploited to disambiguate a vessel orientation with a 180 degree ambiguity and obtain a vessel heading. The systems and methods may be particularly well suited to coarser resolution SAR imagery, such as is typically used for maritime surveillance. At such resolutions, there is not a lot of detail in the SAR vessel signature (set of pixels corresponding to the vessel), so the azimuth smearing may be particularly useful.

[0051] Azimuth smearing is a set of image pixels that are brighter than the ocean background and are adjacent to a ship. The pixels are not focused into anything recognizable, rather they are “smeared” in the azimuth direction (the direction that the SAR satellite is moving). Sometimes, they are a narrow smear, adjacent to one end of the ship, while other times they may extend along the entire length of the ship. The systems and methods described herein leverage the relationship between azimuth smearing and vessel direction of travel to provide an automated approach to determining vessel heading.

[0052] Referring now to Figure 1 , shown therein is a system 100 for vessel detection and characterization from SAR imagery, according to an embodiment.

[0053] The system 100 exploits azimuth smearing in SAR imagery to identify a direction of travel of a vessel 102 relative to a SAR antenna and uses the determined direction of travel to disambiguate a heading of the vessel 102.

[0054] In system 100, a SAR data collector 104, such as a satellite orbiting Earth, acquires SAR imagery of vessels 102 in maritime environments, such as the open ocean or close to land. The SAR data collector 104 includes at least one sensor payload (including a SAR antenna) configured to collect SAR data.

[0055] The SAR data collector 104 transmits the SAR image data to a ground terminal 106 (or ground station 106). The ground terminal 106 may include a receiver and a computer system for processing the received SAR data.

[0056] The system 100 includes a server system 108 in communication with the ground terminal 106 over a communications network 110. The network 110 may be a wide area network, such as the Internet. Communication in this context may include sending and receiving data. The server 108 receives SAR image data from the ground terminal 106. The server 108 may receive the SAR image data through one or more intermediary computer systems.

[0057] The system 100 further includes user device 112 that communicates with the server system 108 via network 110. In some embodiments, instead of or in addition to user device 112, the server system 108 may also communicate with a client device or system that is not a user device (e.g., a computer system running a client application that uses an output of the application 120, described below, such as a vessel heading).

[0058] While a single server computer 108 and a single user device 112 are shown in Figure 1 , the number of servers 108 and user devices 112 may vary (e.g., multiple) and the number is not particularly limited.

[0059] The server 108 runs a vessel detection and characterization software application 120. Characterization in this context includes determining static or dynamic properties of a detected vessel, such as vessel heading. The user device 112 communicates with the server system 108 over the network 110 and provides a user interface of the vessel detection and characterization application 120 for a user to request and review information about vessels automatically detected and characterized by the application 120.

[0060] According to various embodiments, the vessel detection and characterization software application 120 is hosted by the server system 108, or is installed locally on the user device 112, or runs on both the server system 108 and the user device 112.

[0061] The user device 112 is configured to receive input from a user and display data generated by the server 108. The input data received from a user may be used to request certain data generated and stored by the server 108. The user device 112 is configured to display a graphical user interface that allows a user to interact with theserver 108. The user interface may include a series of user interface screens for receiving user input and displaying output data generated by the server 108.

[0062] Referring now to Figure 2, shown therein is a computer system 200 for disambiguating vessel heading, according to an embodiment.

[0063] In an embodiment, the computer system 200 may be implemented using the server system 108 and the user device 112 of Figure 1. It should be noted that, in variations, software modules and components of system 200 may be implemented or executed at a single computing device or across multiple computing devices (e.g., networked computer devices).

[0064] The system 200 includes a display 208 for displaying data generated by the system 100. The display 108 may be located at a user device of the system 200, such as user device 114 of Figure 1 .

[0065] The system 200 includes a memory 202 and a processor 204 in communication with the memory 202.

[0066] The system 200 includes a communication interface 206 for transmitting and receiving data. The communication interface 206 may include a network interface.

[0067] The system 200 includes an input device 210 for providing input data to the system 200 by a user, such as through a graphical user interface. The input device 210 may include a pointing device (e.g., a mouse), a keypad, or the like.

[0068] The processor 204 is configured to execute a vessel detection and characterization application 120 (e.g., vessel detection and characterization application 120 of Figure 1 ).

[0069] The vessel detection and characterization application 120 includes a vessel detection module 212. The vessel detection module 212 receives a SAR image 214 as input and outputs a vessel detection 216. The vessel detection 216 may also be referred to as a “vessel signature”, “SAR signature”, or “SAR vessel signature”. The vessel detection 216 is a set of all pixels in the SAR image that contain backscatter energy from the vessel (or all the pixels belonging to the ship signature). The vessel detection 216 is stored in memory 202.

[0070] In some embodiments, the vessel detection module 212 may output the detection 216 as a SAR image chip. The image chip may be a small region of the SAR image with the detected vessel 216 at the center. For example, the SAR image chip may be 500 x 500 pixels. Image chips may be generated by the vessel detection module 212 as part of a vessel detection report. In some cases, image chips may not be needed if you have the SAR image 214 and know the location of the vessel 216 within the SAR image 214. However, SAR images can be very large and it can be impractical to maintain all the SAR images in storage. It is much more practical to keep just the relatively small portion of the SAR image that includes the vessel 216. In other cases, the output 216 of the vessel detection module 212 is used along with the SAR image 214 to generate a SAR image chip containing the detection 216 (e.g., by a SAR image chip generator module executed by the processor 204). The vessel signature 216 is stored in memory 202. In some cases, image chips may not be needed if you have the SAR image 214 is available and the location of the vessel 216 within the SAR image 214 is known. However, as SAR images can be very large, it may be impractical to maintain all the SAR images in storage. It is much more practical to keep just the relatively small portion of the SAR image that includes the vessel 216 (i.e., the image chip).

[0071] In an embodiment, the vessel detection module 212 uses an adaptive thresholding algorithm, such as Constant False Alarm Rate (“CFAR”), to generate a cluster of bright pixels representing the vessel detection 216. A CFAR detector may compare the intensity of each pixel under test (PuT) to the local background clutter, and if the pixel value exceeds a certain threshold, it is marked as an outlier. Clusters of these outliers are assumed to represent objects of interest.

[0072] The vessel detection and characterization application 120 includes a vessel orientation estimator module 218. The vessel orientation estimator module 218 takes as input a vessel detection 216 and outputs a vessel orientation 220. The vessel orientation 220 is stored in memory 202.

[0073] The vessel orientation 220 includes an angle that the detected vessel is oriented relative to north with a 180-degree ambiguity (i.e., it is not known which end of the vessel is the how and which is the stem). An example vessel orientation 220 is thatthe detected vessel has a heading of 45 / 225 degrees, meaning either 45 or 225 degrees relative to north, with the heading being ambiguous because it is not known which of those two values is the truth. Accordingly, the estimated orientation 220 includes a first heading value 222 and a second heading value 224, which are 180-degrees apart.

[0074] In an embodiment, the vessel orientation estimator module 218 uses weighted Principal Component Analysis, where the weighting uses a function of the backscattered energy. The vessel orientation estimator module 218 may implement a vessel orientation estimation process as described in US Provisional Patent Application No. 63 / 727,256, which is incorporated by reference herein in its entirety.

[0075] The vessel detection and characterization application 120 includes a vessel direction determination module 226. The vessel direction determination module 226 takes as input a vessel signature 216 and outputs a vessel direction of travel 228. The vessel direction of travel 228 is stored in memory 202. The vessel direction of travel 228 indicates whether the detected vessel was headed towards or away from the SAR payload when the SAR image 214 was acquired.

[0076] The vessel direction determination module 226 includes an algorithm that exploits azimuth smearing in the SAR image 214 to determine the vessel direction of travel 228. In some embodiments, the algorithm is machine learning-based. In other embodiments, the algorithm is based on more traditional image processing techniques. Machine learning- and image processing-based embodiments of the vessel direction determination module 226 are described in further detail below.

[0077] The vessel detection and characterization application 120 includes a vessel heading disambiguation module 230. The vessel heading disambiguation module 230 takes as input a vessel orientation 220 and the vessel direction of travel 228 and outputs a disambiguated vessel heading 232 (or simply, vessel heading 232). In doing so, the vessel heading disambiguation module 230 determines which of the possible headings 222, 224 from the estimated vessel orientation is the truth. The disambiguated vessel heading 232 may be specified as angle relative to north. The disambiguated vessel heading 232 is stored in memory 232.

[0078] In some embodiments, the vessel detection and characterization application 120 includes a graphical user interface (GUI) module 234. The GUI module 232 displays the disambiguated vessel heading 232 to the user in a graphical user interface. The GUI module 232 may also display other data stored in memory 202 and status information about the vessel detection and characterization application 120. The GUI module 232 may also receive user inputs, such as user inputs requesting vessel detection and characterization application 120 outputs.

[0079] In some embodiments, the vessel detection and characterization application 120 includes an application programming interface (API) module 236. The API module 236 is used by the vessel detection and characterization application 120 to communicate with other software applications, such as client applications that may use data stored or generated by the application 120. In particular, the vessel detection and characterization application 120 may provide the vessel heading 232 to a client application via the API module 236.

[0080] The memory 202 may also store a vessel width 238. The vessel width 238 may be estimated by the vessel detection and characterization application 120 (e.g., by a width estimator module). The vessel width 238 may be used by the vessel direction determination module 226 when determining vessel direction of travel 228 according to an embodiment that uses image processing techniques, as described herein.

[0081] The memory 202 may also store a vessel length 240. The vessel length 240 may be estimated by the vessel detection and characterization application 120 (e.g., by a length estimator module). The vessel length 240 may be used by the vessel direction determination module 226 when determining vessel direction of travel 228 according to an embodiment that uses image processing techniques, as described herein. The length 240 may be used with the width 238 to define a shape that fits the vessel signature 216 (and which shape is rotated as necessary to the vessel’s orientation).

[0082] An embodiment of vessel direction determination module 226 using machine learning will now be described.

[0083] For the machine learning implementation, a machine learning model may be trained according to method 300 of Figure 3. The method 300, or portions thereof, maybe encoded as computer-executable instructions which, when executed by a processor, cause the processor to perform the method 300. The method 300 may be implemented using the computer system 200 of Figure 2 and, in particular, by the vessel detection and characterization application 120.

[0084] At 302, the method 300 includes providing a set of SAR image chips containing SAR vessel signatures (e.g., vessel signature 216). Some of the image chips contain azimuth smearing and some of the image chips do not contain azimuth smearing. The size of the set is suitable for training the machine learning model to an acceptable performance level. In some cases, the image chips may be extracted from the SAR image in which the vessel was detected.

[0085] At 304, the method 300 includes, for each of the image chips, obtaining an actual heading (i.e. , a numeric heading value) of the vessel in the vessel detection from another source of information (“ground truth”).

[0086] At 306, the method 300 includes, for each vessel in the image chips, determining whether the vessel was headed towards or away from the SAR satellite based on the actual heading from 304.

[0087] At 308, the method 300 includes labelling the image chips with the information from 306 to obtain a set of labelled image chips. The labelling may include appending a label to each image chip indicating whether the vessel in the image chip was heading towards or away from the satellite when the image was acquired.

[0088] At 310, the method 300 includes training a machine learning model using the set of labelled image chips from 308. Any suitable machine learning training process, such as those known in the art, may be used to train the machine learning model. In an embodiment, the machine learning model may be an artificial neural network trained according to a neural network training process.

[0089] The method 300 obtains a trained machine learning model that is used by vessel direction determination module 226. Through the training process, the machine learning model learns that the presence of azimuth smearing is correlated with the desired direction of travel.

[0090] The trained machine learning model is then used to perform inference on new image chips. The trained machine learning model takes a SAR image chip as input and returns a value indicating whether the vessel was headed towards the satellite, or away from the satellite, or unknown. In some embodiments, there may not be a value for unknown category. An unknown category may be useful for image chips that the model was unable to disambiguate the heading for and thus should be left as ambiguous. This value may be the vessel direction of travel 228 (Figure 2). If the machine learning model returns a value that is not “unknown”, that value can be used to disambiguate the heading of the vessel (e.g., disambiguate vessel heading 232 from vessel orientation 220 using the value returned by the machine learning model).

[0091] In some machine learning embodiments, in addition to the model learning the azimuth smearing to help determine direction, the machine learning model may learn from other information in the vessel signature 216 to help with determining the heading 232. For example, the superstructure of a vessel, which is higher up than the deck, is subjected to “layover” where the superstructure is shifted towards the SAR radar more than what the deck is. So, in the example image 400 of Figure 4 (which is an ICEYE image), if a straight line is drawn along the left side of the four ships 402-408, it can be seen that at the bottom of each ship, the bright pixels extend outside (to the left) of the line. The radar is imaging from the left side, so the superstructure “lays over” towards the radar. So, just by looking along the sides of the ship, it can be seen where the superstructure is (as long as the ship is not headed in the along range direction) and for large ships the superstructure is usually at the stern, so for all four of these ships there can be a high confidence that the ships are headed towards the upper right. So, the machine learning approach may be used to learn this information about the superstructure of the vessel as well (plus the superstructure is usually the brightest part of the ship).

[0092] Embodiments of vessel direction determination module 226 using a more traditional image processing approach will now be described.

[0093] In these embodiments, one or more algorithms are explicitly developed to detect whether there was azimuth smearing near a SAR vessel signature 214, and on which side.

[0094] To do this, the algorithm starts with the vessel orientation 220 that has been determined from the SAR vessel signature 216.

[0095] The vessel direction determination module 226 analyzes the image pixels in the vessel signature 216 in regions offset from the ship. The vessel direction determination module 226 considers more than just the ship pixels, because azimuth smearing typically is brighter than the ocean pixels, but often not bright enough to be considered part of the ship.

[0096] Figure 5 shows an example image chip 500 of a ship with azimuth smearing, according to an embodiment. Looking at the image 500, there is no way to determine which of the two possible ways the ship is headed.

[0097] Figure 6 shown the image chip 500 of Figure 5 divided into different regions. The dividing of image chip 500 into regions is performed by processor 204, such as by vessel direction determination module 226.

[0098] Regions 1 and 2 are the regions in front of and behind the ship and no azimuth smearing would occur in those regions.

[0099] Regions 3 and 4 are the regions where azimuth smearing could occur.

[0100] Using the estimates of vessel orientation 220, vessel width 238, and vessel length 240, the vessel direction determination module 226 excludes the actual ship pixels from regions 3 and 4. This may be done, for example, by defining a shape (e.g., a rectangle) that fits around the vessel signature 216 based on the estimated width and length 238, 240 and rotating the defined shape to the orientation of the vessel.

[0101] When the algorithm executed by the vessel direction determination module 226 indicates that azimuth smearing is present in either region 3 or region 4, that information is used to disambiguate the heading 232 of the vessel (e.g., using vessel heading disambiguation module 230). In this case, the indication of azimuth smearing present in region 3 or region 4 may considered the vessel direction of travel 228 or thevessel direction of travel 228 may be inferred by the vessel direction determination module 226 based on the region that contains azimuth smearing.

[0102] Further details of particular image processing-based embodiments will now be described, with reference to Figure 6.

[0103] In a first embodiment, for detecting vessel signatures 216 (e.g., by module 212), a model of ocean returns is used. A threshold value called a “CFAR threshold” is determined from the model. Pixels above the threshold value are considered to be part of the SAR vessel signature 216. Using the same model, a second, lower threshold is defined. The second threshold may be referred to as a “smearing threshold”. The vessel direction determination module 226 looks for pixels in regions 3 and 4 that are above the smearing threshold but below the CFAR threshold. The objective is for the vessel direction determination module 226 to capture the pixels that make up the azimuth smearing. This could, however, include some ocean pixels that are simply brighter than usual. As those ocean pixels would generally be isolated pixels, the vessel direction determination module 226 uses a grouping or region growing algorithm to find groups of pixels that are adjacent to each other. Based on these groupings, the vessel direction determination module 226 determines whether azimuth smearing is present in region 3 or region 4, or in neither region. The presence of azimuth smearing is then used by the vessel direction determination module 226 to infer and output the vessel direction of travel 228 (e.g., towards the satellite, away from the satellite, unknown).

[0104] In a second embodiment, the vessel direction determination module 226 uses a statistical test to determine whether region 3 or region 4 is statistically different from regions 1 and 2. In a simple implementation, the test may compare the mean pixel value of the regions 1 -4. Regions 1 and 2 are combined into a single region (region 1 +2). Region 3 is compared against region 1 +2. Region 4 is compared against region 1 +2. Region 3 is compared against region 4. The vessel direction determination module 226 determines that azimuth smearing occurred in region 3 is the mean pixel value of region 3 is greater than the mean pixel value of region 1 +2 and the mean pixel value of region 4. Likewise, azimuth smearing is identified in region 4 if the mean pixel value of region 4 is greater than the mean pixel value of region 3 and the mean pixel value of region 1 +2.In comparing mean pixel values, the vessel direction determination module 226 may compare the difference to a threshold (mean pixel value difference threshold) and require the mean pixel value difference to exceed the threshold to identify azimuth smearing. Further, in other embodiments, the vessel direction determination module 226 may use more complex statistical tests than comparing mean pixel values. In some embodiments, techniques developed for image segmentation in SAR images may be adapted to test whether the regions are different.

[0105] While the above description provides examples of one or more apparatus, methods, or systems, it will be appreciated that other apparatus, methods, or systems may be within the scope of the claims as interpreted by one of skill in the art.

Claims

Claims:

1. A system for determining vessel heading in synthetic aperture radar (SAR) imagery, the system comprising: a computer memory that stores: a vessel signature comprising a set of SAR image pixels corresponding to a vessel detected in a SAR image containing the set of SAR image pixels; and an orientation of the vessel, the orientation having a 180 degree ambiguity; one or more processors in communication with the computer memory and configured to: process the vessel signature with a vessel direction determination module configured to determine a direction of travel of the vessel relative to a SAR payload that captured the SAR image based on azimuth smearing; and determine a heading of the vessel by disambiguating the orientation using the determined direction of travel.

2. The system of claim 1 , wherein the vessel direction determination module executes an algorithm that exploits azimuth smearing in the SAR image to determine the direction of travel.

3. The system of claim 2, wherein the algorithm: identifies pixel regions in the SAR image relative to the vessel signature where azimuth smearing could occur, wherein the identifying includes excluding from the pixel regions a set of actual vessel pixels;determines that azimuth smearing is present in at least one of the pixel regions; and determines the direction of travel of the vessel based on a location of the at least one pixel region with azimuth smearing relative to the vessel.

4. The system of claim 3, wherein the algorithm excludes the set of actual vessel pixels by excluding pixels within a shaped defined using an estimated length and width of the vessel and the orientation of the vessel.

5. The system of claim 3, wherein identifying the pixel regions includes excluding pixel regions in front of the vessel and behind the vessel.

6. The system of claim 5, wherein the algorithm determines that azimuth smearing is present by comparing the pixel regions where azimuth smearing could occur to the excluded pixel regions.

7. The system of claim 3, wherein the algorithm determines that azimuth smearing is present by comparing pixel values of the pixel regions where azimuth smearing could occur to a smearing threshold and identifies azimuth smearing in the pixel regions when the pixel values exceed the smearing threshold.

8. The system of claim 7, wherein the smearing threshold is set lower than a threshold used to identify the vessel signature.

9. The system of claim 7, wherein the pixel values are pixel brightness values.

10. The system of claim 5, wherein the algorithm determines that azimuth smearing by determining whether the pixel regions are statistically different from the excluded pixel regions.11 . The system of claim 2, wherein the algorithm:identifies pixel regions in the SAR image relative to the vessel signature where azimuth smearing could occur, where those pixel regions exclude a set of actual vessel pixels; and determines the direction of travel of the vessel based on which of the pixel regions contains azimuth smearing.

12. The system of claim 2, wherein the algorithm is a machine learning model.

13. The system of claim 12, wherein the machine learning receives an image chip containing the vessel signature as input and returns a value indicating the direction of travel of the vessel.

14. The system of claim 3, wherein the direction of vessel travel is away from the SAR payload or towards the SAR payload.

15. The system of claim 12, wherein the machine learning model is trained using a method comprising: providing a set of SAR image chips containing vessel signatures, the set including SAR image chips with azimuth smearing and SAR image chips without azimuth smearing; determining, for each vessel signature, whether the vessel was headed towards or away from the SAR payload using a known heading of the vessel; labelling the SAR image chips in the set by appending a label indicating whether the vessel in the SAR image chip was heading towards or away from the SAR payload; and training the machine learning model using the labelled set of SAR image chips.

16. A method of determining vessel heading in synthetic aperture radar (SAR) imagery, the method comprising: storing, in a data storage device: a vessel signature comprising a set of SAR image pixels corresponding to a vessel detected in a SAR image containing the set of SAR image pixels; and an orientation of the vessel, the orientation having a 180 degree ambiguity; processing the vessel signature with a vessel direction determination module configured to determine a direction of travel of the vessel relative to a SAR payload that captured the SAR image based on azimuth smearing; and determining a heading of the vessel by disambiguating the orientation using the determined direction of travel.

17. The method of claim 16, wherein the vessel direction determination module executes an algorithm that exploits azimuth smearing in the SAR image to determine the direction of travel.

18. The method of claim 17, wherein the algorithm: identifies pixel regions in the SAR image relative to the vessel signature where azimuth smearing could occur, wherein the identifying includes excluding from the pixel regions a set of actual vessel pixels; determines that azimuth smearing is present in at least one of the pixel regions; anddetermines the direction of travel of the vessel based on a location of the at least one pixel region with azimuth smearing relative to the vessel.

19. The method of claim 18, wherein the algorithm excludes the set of actual vessel pixels by excluding pixels within a shaped defined using an estimated length and width of the vessel and the orientation of the vessel.

20. The method of claim 18, wherein identifying the pixel regions includes excluding pixel regions in front of the vessel and behind the vessel.21 . The method of claim 20, wherein the algorithm determines that azimuth smearing is present by comparing the pixel regions where azimuth smearing could occur to the excluded pixel regions.

22. The method of claim 18, wherein the algorithm determines that azimuth smearing is present by comparing pixel values of the pixel regions where azimuth smearing could occur to a smearing threshold and identifies azimuth smearing in the pixel regions when the pixel values exceed the smearing threshold.

23. The method of claim 22, wherein the smearing threshold is set lower than a threshold used to identify the vessel signature.

24. The method of claim 22, wherein the pixel values are pixel brightness values.

25. The method of claim 20, wherein the algorithm determines that azimuth smearing is present by determining whether the pixel regions are statistically different from the excluded pixel regions.

26. The method of claim 17, wherein the algorithm:identifies pixel regions in the SAR image relative to the vessel signature where azimuth smearing could occur, where those pixel regions exclude a set of actual vessel pixels; and determines the direction of travel of the vessel based on which of the pixel regions contains azimuth smearing.

27. The method of claim 17, wherein the algorithm is a machine learning model.

28. The method of claim 27, wherein the machine learning receives an image chip containing the vessel signature as input and returns a value indicating the direction of travel of the vessel.

29. The method of claim 18, wherein the direction of vessel travel is away from the SAR payload or towards the SAR payload.

30. The method of claim 27, wherein the machine learning model is trained using a method comprising: providing a set of SAR image chips containing vessel signatures, the set including SAR image chips with azimuth smearing and SAR image chips without azimuth smearing; determining, for each vessel signature, whether the vessel was headed towards or away from the SAR payload using a known heading of the vessel; labelling the SAR image chips in the set by appending a label indicating whether the vessel in the SAR image chip was heading towards or away from the SAR payload; and training the machine learning model using the labelled set of SAR image chips.1 . A method of determining vessel direction of travel in SAR imagery using azimuth smearing, the method comprising: providing a set of SAR image chips containing vessel signatures, the set including SAR image chips with azimuth smearing and SAR image chips without azimuth smearing; determining, for each vessel signature, whether the vessel was headed towards or away from the SAR payload using a known heading of the vessel; labelling the SAR image chips in the set by appending a label indicating whether the vessel in the SAR image chip was heading towards or away from the SAR payload; training the machine learning model using the labelled set of SAR image chips; and using the trained machine learning model to determine a direction of travel of a vessel detected in a SAR image.

Images