System and method for detecting and tracking rsos using optical imagery for space domain awareness

Abstract

Systems and methods for detecting and tracking resident space objects (RSOs) are provided. In an embodiment, the system uses an optical sensor on some or all spacecraft in a constellation to track RSOs for space domain awareness. The system performs RSO detection and tracking onboard individual satellites using data collected by the respective sensors and performs further RSO detection and tracking using outputs from multiple spacecraft. In an embodiment, the optical sensor may be a wide field of view optical sensor, such as a star tracker sensor. The star tracker sensor may be a repurposed star tracker sensor configured to operate in a different manner from its typical or traditional star tracking function.

Description

SYSTEM AND METHOD FOR DETECTING AND TRACKING RSOs USING OPTICAL IMAGERY FOR SPACE DOMAIN AWARENESSTechnical Field

[0001] The following relates generally to satellite constellations, and more particularly to systems and methods for space domain awareness using satellite constellations.Introduction

[0002] As the number of satellites launched into space grow exponentially, space situational awareness (SSA) and space traffic management (STM) are becoming increasingly a concern to ensure safe operations. The ability to detect and track resident space objects (RSOs) - natural or artificial objects that orbit another body - is an important component of SSA and STM.

[0003] Accordingly, there is a need for an improved system and method for detecting and tracking RSOs that overcomes at least some of the disadvantages of existing systems and methods.Summary

[0004] The present disclosure provides systems and methods for detecting and tracking resident space objects (RSOs).

[0005] A method of space domain awareness is provided. The method includes: collecting optical image data of a space environment with an optical image sensor onboard a satellite; and obtaining resident space object (RSO) data describing an RSO in the optical image data using a processor onboard the satellite. The obtaining includes detecting the RSO in the optical image data and determining tracking information for the RSO.

[0006] In some embodiments, the method further includes: performing the collecting the optical image data and the obtaining RSO data on a plurality of satellites in a satellite constellation; and combining the RSO data from the plurality of satellites to obtain enhanced RSO data for the RSO.

[0007] In some embodiments, the tracking information comprises data about position and velocity of the RSO over time and is used to predict where the RSO will be in the future.

[0008] In some embodiments, the optical image data comprises a tracklet of images, and detecting the RSO in the optical image data comprises: determining image coordinates of the RSO in each image in the tracklet; converting the image coordinates to real world coordinates; performing an initial orbit determination using the real world coordinates, timestamp data, and magnitude data; calculating a pseudo range indicating an altitude the RSO would be at if the initial orbit determination was correct; refining the initial orbit determination using coordinates from other observations; and identifying refined orbit parameters as final orbit parameters once an expected location of the RSO is sufficiently close to an observed location of the RSO, wherein a determination of whether the expected location and the observed location are “sufficiently close” is determined by the processor according to one or more stored criteria.

[0009] In some embodiments, refining the initial orbit determination is performed iteratively until either a maximum number of cycles is achieved or a threshold score has been reached.

[0010] In some embodiments, the method is performed with additional tracklets from at least one other satellite providing a different viewing angle.

[0011] In some embodiments, the method further includes performing enhanced streak and spot detection, comprising: stacking images from a tracklet along with a coordinate list of objects from each image; tracking motion of each object across the stack of images, using Euclidean distance and motion fitting; generating a list of locations for each object; identifying, using motion filtering, the objects that move like stars as stars and the objects that do not move like stars as RSOs; creating an RSO list listing centroid locations of the objects identified as RSO; and creating a star list listing centroid locations of the objects identified as stars.

[0012] In some embodiments, the RSO data includes detected streaks or spots in the optical image data, the detected streaks or spots representing an angular distance that a detected object traveled over integration time, and the detected streaks or spots are used to distinguish the RSO from starts in the optical image data.

[0013] In some embodiments, detecting the RSO includes distinguishing the RSO from stars in the optical image data using measured streak or spot data.

[0014] In some embodiments, distinguishing the RSO from stars includes identifying that the RSO presents as a streak with a different length or orientation from what is expected for a star.

[0015] In some embodiments, detecting the RSO in the optical image data includes using a spot or streak detection algorithm to identify and characterize spots or streaks in the optical image data.

[0016] In some embodiments, the method further includes calibrating an optical response of the optical image sensor, the calibrating comprising: obtaining observations corrections data using reference data from a reference library; and updating the optical response of the optical image sensor using the corrections data.

[0017] In some embodiments, the method further includes updating local RSO tracking information stored onboard the satellite with the RSO tracking information or transmitting the RSO tracking information to a ground system or other satellite.

[0018] In some embodiments, the detected RSO is an RSO that is already being tracked by the processor as existing tracking information for the RSO and the RSO tracking information is used to update the existing tracking information.

[0019] In some embodiments, the detected RSO is identified by the processor as a newly detected RSO that is not already being tracked by the processor.

[0020] In some embodiments, the optical image data includes first optical image data collected by a first optical image sensor on a first satellite and second optical image data collected by a second optical image sensor on a second satellite, the first and second optical image data are contemporaneous observations of the RSO from the at least two satellites, and obtaining the RSO data includes performing the detecting and determining on the first and second optical image data on the first and second satellites, respectively, and fusing the tracking information obtained by the first and second satellites.

[0021] In some embodiments, the method further includes combining the tracking information of the RSO with additional tracking information of the RSO obtained from at least one other satellite to obtain enhanced tracking information for the RSO.

[0022] In some embodiments, the combining is performed on the satellite, on the at least one other satellite, or at a ground system.

[0023] In some embodiments, the optical sensor is a star tracker sensor that has been reconfigured to observe RSOs instead of stars.

[0024] A system is also provided that is configured to implement any of the foregoing methods. The system includes the optical image sensor and the processor.

[0025] A system for space domain awareness is also provided. The system includes: an optical image sensor disposed on a satellite, the optical image sensor configured to collect optical image data of a space scene; a processor disposed on the satellite, the processor configured to obtain resident space object (RSO) data describing an RSO in the optical image data, the obtaining comprising: detecting the RSO in the optical image data; and determining tracking information for the RSO; and a communication unit disposed on the satellite and configured to transmit or receive the RSO data to or from a system external to the satellite.

[0026] In some embodiments, the system further includes: a second optical image sensor disposed on a second satellite, the second optical image sensor configured to collect second optical image data of a space scene; a second processor disposed on the second satellite, the second processor configured to obtain second resident space object (RSO) data describing the RSO in the second optical image data, the obtaining comprising: detecting the RSO in the second optical image data; and determining second tracking information for the RSO; a second communication unit disposed on the second satellite configured to transmit or receive the second RSO data to or from a second system external to the second satellite; and a combining module configured to combine the tracking information and the second tracking information to obtain enhanced tracking information for the RSO.

[0027] In some embodiments, the combining module is implemented on the satellite or the second satellite, and the communication unit and the second communication unit use an optical intersatellite link.

[0028] In some embodiments, the combining module is implemented at a ground system, and the satellite and the second satellite downlink the tracking information and the second tracking information to the ground station via the communication unit and the second communication unit, respectively.

[0029] In some embodiments, the optical sensor is a star tracker sensor that has been reconfigured to observe RSOs instead of stars.

[0030] Other aspects and features will become apparent, to those ordinarily skilled in the art, upon review of the following description of some exemplary embodiments.Brief Description of the Drawings

[0031] The drawings included herewith are for illustrating various examples of articles, methods, and apparatuses of the present specification. In the drawings:

[0032] Figure 1 is a schematic diagram of a system for resident space object (“RSO”) detection and tracking, according to an embodiment;

[0033] Figure 2 is a block diagram of components of the system of Figure 1 in further details, according to an embodiment;

[0034] Figure 3 is a flowchart of a method of RSO detection and tracking, according to an embodiment;

[0035] Figure 4 is a flowchart illustrating a multimodal input process that may be used by the RSO detection and tracking systems and methods of the present disclosure, according to an embodiment;

[0036] Figure 5 is a flowchart illustrating the orbit refinement of Figure 4 in further detail, according to an embodiment.Detailed Description

[0037] Various apparatuses or processes will be described below to provide an example of each claimed embodiment. No embodiment described below limits any claimed embodiment and any claimed embodiment may cover processes or apparatuses that differ from those described below. The claimed embodiments are not limited to apparatuses or processes having all of the features of any one apparatus or process described below or to features common to multiple or all of the apparatuses described below.

[0038] One or more systems described herein may be implemented in computer programs executing on programmable computers, each comprising at least one processor, a data storage system (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device. For example, and without limitation, the programmable computer may be a programmable logic unit, a mainframe computer, server, personal computer, cloud-based program or system, laptop, personal data assistance, cellular telephone, smartphone, space-based computer or tablet device.

[0039] Each program is preferably implemented in a high-level procedural or object-oriented programming and / or scripting language to communicate with a computer system. However, the programs can be implemented in assembly or machine language,if desired. In any case, the language may be a compiled or interpreted language. Each such computer program is preferably stored on a storage media or a device readable by a general or special purpose programmable computer for configuring and operating the computer when the storage media or device is read by the computer to perform the procedures described herein.

[0040] A description of an embodiment with several components in communication with each other does not imply that all such components are required. On the contrary, a variety of optional components are described to illustrate the wide variety of possible embodiments of the present invention.

[0041] Further, although process steps, method steps, algorithms or the like may be described (in the disclosure and I or in the claims) in a sequential order, such processes, methods and algorithms may be configured to work in alternate orders. In other words, any sequence or order of steps that may be described does not necessarily indicate a requirement that the steps be performed in that order. The steps of processes described herein may be performed in any order that is practical. Further, some steps may be performed simultaneously.

[0042] When a single device or article is described herein, it will be readily apparent that more than one device I article (whether or not they cooperate) may be used in place of a single device I article. Similarly, where more than one device or article is described herein (whether or not they cooperate), it will be readily apparent that a single device I article may be used in place of the more than one device or article.

[0043] The following relates generally to satellite constellations, and more particularly to systems and methods for space domain awareness using satellite constellations.

[0044] The present disclosure provides a system that uses an optical sensor (e.g., camera), also referred to as an optically-based payload, on some or all spacecraft on a satellite constellation with an intersatellite link that operate as a network of optical sensors observing the space environment and the RSOs therein. The optical image data is processed onboard the spacecraft and on ground to track RSOs for space domain awareness. In a particular embodiment, the optical camera is a dedicated star tracker. In general, the camera is not pointed or steered but rather used to collect optical image data of whatever falls within its field of view (FOV). For example, instead of introducing atasking list to the camera, the camera may be configured to be always looking, interpreting, and segmenting the scene.

[0045] The systems and methods may be used to track debris and other objects that may or may not be tracked by national defense agencies or commercial entities.

[0046] In an embodiment, the systems and methods use optical image data captured by an optical sensor. The optical sensor may be a wide FOV sensor. The optical sensor may be a star tracker sensor. The star tracker sensor is configured to operate in a different manner from its typical or traditional method of operation (star tracking). The star tracker sensor is re-configured to perform a space domain awareness function (as opposed to, or in addition to, a star tracker function). The space domain awareness function may be encoded as computer-executable instructions stored in a data storage device and executed by a processor (e.g., a processor of the optical sensor, or separate from the optical sensor). In general, the space domain awareness function is configured to collect a lot of signal (significantly more than would be collected with a star tracker function). The star tracker sensor may be configured to have longer exposure times than for normal or standard star tracking by the star tracker sensor to capture dimmer objects (than would be captured under normal star tracker settings).

[0047] The systems and methods process optical image data collected by the optical sensor. The optical image data is processed to obtain all the RSOs in the scene (RSO data). Processing may include segmenting the image data into all the RSOs in the scene. The RSO data is then further processed using one or more image processing algorithms.

[0048] Processing of output data generated onboard multiple spacecraft in the constellation may then be mosaiced or combined through image processing techniques to obtain a single picture of RSO detection and tracking. By using multiple geometries from the different spacecraft in the constellation, the RSO’s change in angular position relative to the stars can be used to help reduce false positive detections of RSO’s. Further, the different angular positions and uncertainties can be fused to refine the position and reduce the uncertainty of the angular position and range of the RSO (calculates range and range-rate measurements creating more precision in state vectors and position propagation). This processing may be performed on ground, using edge computing, onboard one or more SSA satellites, or some combination thereof. In some embodiments, this processing may be performed completely onboard via intersatellite links and sharing the compute workload amongst the cooperating network of opticalsensors observing the same RSO. In other embodiments, this processing may be space cloud-based where the network of SSA satellites network via a satcom service provided to a cloud provider and return the results to either the SSA satellite or to a ground station.

[0049] The term “tracking information” as used herein refers to data about a satellite’s position and velocity overtime and is used to predict where a satellite will be in the future. Tracking information can be made of position and velocity vectors and / or orbital elements (which are defined in Two Line Elements (TLE)s).

[0050] Referring now to Figure 1 , shown therein is a system 100 for space domain awareness, according to an embodiment. The system 100 may be used to detect and track resident space objects (“RSOs”) 110 in optical image data.

[0051] The system 100 includes a space segment 102 and a ground segment 104.

[0052] The space segment 102 includes a plurality of spacecraft 106. The plurality of spacecraft 106 may be a constellation, such as a communication, earth observation, or other type of satellite constellation. The plurality of spacecraft 106 may include an intersatellite link (ISL) allowing spacecraft in the constellation to link to one another and relay data in space (i.e., without having to relay to ground). A single spacecraft 106 is shown and described for simplicity, but it will be understood that implementations including multiple, even hundreds or more spacecraft 106 are contemplated.

[0053] The spacecraft 106 collects optical image data of the environment using one or more onboard optical imaging sensors. The optical image sensor may have a wide field of view (WFoV) and, in such case, may be referred to as a WFoV sensor. The optical imaging sensor collects image data in its field of view. The optical image data is processed by the system 100 to detect and track RSOs 110. The required FoV of the optical image sensor may be derived as an optimal value depending on a number of factors dictating the required coverage from an individual sensor. Factors may include, for example, the size of constellation (i.e., number of sensors), minimum revisit frequency, etc. However, a minimum FoV value of the optical image sensor may be dictated by the requirement that an entire RSO streak length will be included in a single image. In some embodiments, the optical image sensor has an FoV value in the range of 15 to 80 degrees. In some embodiments, the optical sensor may be an additional payload in a satcom constellation.

[0054] The spacecraft 106 also includes an onboard processing unit 108. The onboard processing unit 108 executes an RSO detection and tracking softwareapplication. The RSO detection and tracking application is configured to receive the collected optical image data as input and process the optical image data to detect streaks and spots in the image data (detected streak and spot data). Generally, the processing by the OBP unit 108 reduces the amount of data that is downlinked to ground 104.

[0055] The spacecraft 106 communicates with a ground system or station 112 (or ground terminal) via an uplink 114 and downlink 116. In a satcom system, the ground station 112 is a gateway to an Internet backbone. The manner of communication is generally known. In other embodiments, there may be a plurality of ground systems 112 and the number of ground systems 112 is not particularly limited. The ground systems may be located in multiple geographic locations.

[0056] The ground system 112 includes an antenna system 118 and a data processing device 120. The ground system 112 communicates with the spacecraft 106 via the antenna system 118. The ground system 112 receives an output of the RSO detection and tracking application from the spacecraft 106.

[0057] The data processing device 120 may be configured to store and format received output data of the OBP unit 108.

[0058] The ground segment 104 further includes a user device 128. The user device 128 is configured to receive input from a user and display data generated by the OBP 108 and the data processing device 120. The user device 128 is configured to display a graphical user interface that allows a user to interact with the data generated by the OBP 108 and the data processing device 120. The user interface may include a series of user interface screens for receiving user input and display output data generated by the data processing device 120.

[0059] The user device 128 and the data processing device 120 communicate via a network 130. The network 130 may be a wide area network, such as the Internet. Communication in this context may include sending and receiving data.

[0060] The ground system 104 further includes a space surveillance network 132. The space surveillance network 132 may provide or implement a commercial operational service, a space traffic management system, a space surveillance network, or the like. In some cases, rather than or in addition to communication with user device, data processing device 120 may communicate with space surveillance network 132 via network 130. For example, the data processing device 120 may generate output data as described herein and provide the output data to the space surveillance network 132. Thespace surveillance network 132 may also be referred to as a client system 132 that consumes the data generated by the data processing server 120.

[0061] In an embodiment, the system 100, and in particular spacecraft 106 and data processing device 120, perform RSO detection and tracking, enhanced RSO tracking by triangulation, combining or mosaicking RSO detection outputs from multiple satellites, and enhanced streak and spot detection.

[0062] RSO detection and tracking is performed by a processor on satellite 106 and may be performed as follows. Given an onboard optical sensor (e.g., sensor 108), several images are taken in series - this is called a tracklet. The processor determines the location of RSOs from each image in the tracklet and then perform centroiding. Centroiding is a technique used to find the location of the center of the RSO blob. From there, given the location of the satellite and the boresight of the camera 108, the processor converts the coordinates from image coordinates to real world coordinates (right ascension (RA) and declination (Dec)). Using the RA, Dec, time stamp, and magnitude over the tracklet of images, the processor performs initial orbit determination. From this, the processor calculates a pseudo range, which is what the altitude of the spacecraft would be at the time if the original orbit estimation was correct. The processor then incorporates coordinates from more observations into a calculation called “orbit refinement”. This is iteratively trying to come up with a better mathematical representation of the orbit given more real-world datapoints. The processor identifies final orbit parameters once the expected location of an RSO is sufficiently close to the observed location of said RSO. The foregoing process of RSO detection and tracking may be implemented by module 212 of Figure 2, described below.

[0063] Enhanced RSO tracking by triangulation is performed by a processor at data processing device 120 (though, in other embodiments, this may be performed onboard one or more satellites in a constellation). The enhanced RSO tracking by triangulation works the same way as RSO detection and tracking, above, but with more image tracklets (i.e. , more datapoints) from different angles (as image tracklets may be received from multiple satellites). This generally means that the error in the calculations is smaller and a more accurate orbit can be obtained faster. The foregoing process of enhanced RSO tracking by triangulation may be implemented by module 240 of Figure 2.

[0064] Combining or mosaicking RSO detection outputs from multiple satellites in a constellation is performed by a processor at data processing device 120 (though, inother embodiments, this may be performed onboard one or more satellites in a constellation). Combining or mosaicking the data in this way can provide a single picture of RSO detection and tracking. This process may be performed in the same manner as enhanced RSO tracking by triangulation. Triangulating satellites and combining / mosaicking outputs from different satellites may be considered the same thing. Figure 4 illustrates a process workflow for combining or mosaicking RSO detection outputs, according to an embodiment. Figure 4 represents a multimodal system with another type of data. If range data from a sensor like a ground based radar dish is included, the highly error-prone altitude estimate from optical sensors can be eliminated. This can speed up the orbit determination / refinement process.

[0065] Enhanced streak and spot detection is performed by a processor at data processing device 120 (though, in other embodiments, this may be performed by a processor onboard one or more satellites in a constellation). In an embodiment, this may be performed as follows. First, the processor stacks images from a tracklet, along with a coordinate list of objects from each image. Using a process called Euclidean distance and motion fitting, the processor tracks the motion of each specific object across the stack of images. The processor generates a list of the locations for each individual object. Up to this point, this process is called Object Custody and Tracking. Next, the processor creates an RSO and star centroid list. Using motion filtering, the processor determines which of the individual objects are moving in the same kind of way (like stars) and which of the objects are moving differently (like the RSOs). The processor generates a list of the centroid location of each type. If there was one RSO in the picture, the output here would be two lists. One list of the centroid locations of things that all move like stars, and the other list would have the centroid locations of all the RSO sightings. The foregoing process of enhanced streak and spot detection may be implemented by the module 236 of Figure 2.

[0066] Referring now to Figure 2, shown therein are components of the system 100 of Figure 1 in further detail, according to an embodiment.

[0067] Spacecraft 106 includes onboard processing unit (OBP) 108.

[0068] OBP 108 executes an RSO detection and tracking software application 208. The RSO detection and tracking application 208 receives optical image data as input and generates RSO detection and tracking data as output. The RSO detection and tracking data includes detected streaks and spots in the optical image data. Streaks and spots in the context of RSO detections represent an angular distance that a detected object (theRSO) traveled over the integration time. In the context of the stare mode being used, both stars and RSOs would appear as streaks. RSOs that have spectral reflections (glints) will appear as dots depending on the duration of the glints. Due to this, both spots and streaks are required and used to determine and differentiate RSOs from stars in the image. If a shorter integration time is used, the streaks will get smaller until they are spots requiring both spot and streak identification.

[0069] The RSO detection and tracking application 208 includes an observations correction module 210, an RSO detection and tracking module 212, and a data compression module 214.

[0070] Spacecraft 106 also includes an onboard computer (OBC) 209.

[0071] The OBP 210 is interfaced to the onboard computer 209. The OBP 210 receives various data from the onboard computer 209 including commands (e.g., images to acquire) and host satellite 106 specific data (e.g., host spacecraft 106 position, velocity, attitude, slew rates, etc.).

[0072] Spacecraft 106 includes an optical imaging sensor 202.

[0073] The optical imaging sensor 202 may be a star tracker sensor. The star tracker may be reconfigured from a star tracker functionality to a space domain awareness functionality for it use in system 100.

[0074] The optical imaging sensor 202 collects optical image data 218 in its field of view or environment. Collected optical data 218 includes optical data of reference stars 226 and optical data of RSO targets 228, as described below. Optical data of reference stars 218 may also be referred to as calibration data 218. Optical data of RSO targets 228 may also be referred to as RSO data 228.

[0075] The RSO data 228 is the measured spot or streak image data for RSO targets. The system 100 may detect non-star objects (e.g., RSOs) in an image by filtering for streaks in the image that are of a different length or orientation to what would be expected for stars (as derived from the known motion of the sensor 202).

[0076] Spacecraft 106 includes a sensor control data flow 229 between the optical sensor 202 and the OBP 108.

[0077] The sensor control data flow 229 allows the OBP 108 to exert command and control over the optical sensor 202. This may include, for example, modifying settings, controlling the electronic shutter, choice of sub-frames, binning, masking, etc..For example, sensor 202 control may be used when the star tracker 202 is being used to detect targets in a direction a-priori known to have higher background noise (e.g., zodiacal glow) when the camera setting (exposure, gain, etc.) may be adjusted to optimize performance.

[0078] Spacecraft 106 also includes a reference library 206 stored in one or more data storage devices in communication with the OBP 108. The reference library 206 stores reference data 224 used by the RSO detection and tracking application 208. The reference data 224 is provided as input to the observations corrections module 210 of the OBP.

[0079] The reference library 206 may include any one or more of calibration star catalogues, RSO spectral libraries, and other reference databases.

[0080] A calibration star catalogue is a catalogue or database of stars of a-priori known optical characteristics (e.g., the Hipparcos catalogue). These a-priori known optical characteristics may be referred to as reference star calibration data (which may be a component of reference data 224). The reference star calibration data is provided as reference data 224 input to the observations corrections module 210 of the OBP 108.

[0081] Stars selected from the calibration star catalogue have their photometric data (spots or streaks) collected periodically by the optical sensor 202 so that the onboard optical sensor 202 instrument photometric and astrometric responses can be checked by comparison of the measured magnitude and location against the a-priori known reference data 224 from the calibration star catalogue. Photometric response relates to the accuracy of the brightness measurements produced by the sensor 202. Astrometric response relates to the accuracy of the position measurements produced by the sensor 202. This collected optical data may be referred to as reference star measured data 226.

[0082] The measured reference star data 226 is provided as input to the observations correction module 210.

[0083] The observations correction module 210 compares the received reference star measured data 226 against the reference data 224 (e.g., reference star calibration data from a calibration star catalogue) to check the optical response of the optical sensor 202.

[0084] The observations correction module 210 applies various corrections to the measured image data derived from the various sources including “flat” images and the corrections for instrument response.

[0085] The output of the comparison by the observations corrections module 210 is corrections data.

[0086] Once the comparison is done and corrections data obtained, any required corrections to the optical response of the instrument 202 can be updated using the corrections data before optical measurements are taken from RSO targets by the optical sensor 202. Corrections to the measured RSO data 228 may be applied in the observations correction module 210.

[0087] Once the optical response of the optical sensor 202 has been calibrated using the reference data 224 and reference star measured data 226, the optical sensor 202 collects optical data of RSO targets (measured RSO target data 228).

[0088] The measured RSO data 228 is provided as input to the observations correction module 210.

[0089] The observations correction module 210 applies various corrections to the measured RSO image data 228 derived from the various sources including “flat” images and the corrections for instrument response. Flats are used to ensure that the pixel responses are similar across the entire field of view (FoV). Corrections for instrument response compensate for differences in detector sensitivity as a function of wavelength. The aim is that the corrected measured data 230 should more clearly show the impact of the reflectance of the materials making up the RSOs once all the other factors are compensated for.

[0090] The observations correction module 210 outputs corrected measured RSO data 230.

[0091] The corrected measured RSO data 230 is provided as input to the RSO detection and tracking module 212 and to the data compression module 214.

[0092] The RSO detection and tracking module 212 receives the corrected measured RSO data 230 as input and generates detected streak and spot data 232 as output. In some embodiments, the RSO detection and tracking module 212 may use one more off-the-shelf algorithms, such as Canny Edge Detection and Hough Transform, Radon Transform, or ASTRIDE Boundary Tracking.

[0093] The RSO detection and tracking module 212 takes the corrected measured image data 230 and, using a spot and streak detection algorithm, identifies andcharacterizes the particular spots and streaks 232 present in the corrected measured image data 230.

[0094] The spot and streak algorithm may include a machine learning algorithm. The machine learning algorithm may be trained on a set of real or simulated images. The trained machine learning algorithm is configured to use the shape of the spot / streak to determine if it is an RSO or star.

[0095] The spot and streak algorithm may include an analytical algorithm. The analytical algorithm may vary depending on whether spots or streaks are being used.

[0096] In an embodiment, an analytical spot algorithm may be used if spots are being used. The algorithm stacks multiple images together (e.g., three or more) and tracks objects across the image frames to create a pseudo streak. Once the pseudo streak is obtained, the spot algorithm uses an analytical streak algorithm, such as described below, on the pseudo streaks to differentiate between stars and RSOs.

[0097] In an embodiment, an analytical streak algorithm uses the fact that stars are stationary, so the streak is generated primarily by the attitude change. All the stars’ streaks in the image should be similar in direction and size. The streaks are compared, and outlier streaks are identified. Outlier streaks are tagged as RSOs. The different direction and size of streak from the RSO is caused by its position change over the integration time.

[0098] To increase fidelity, or where there are a lack of stars in the image, the attitude of the spacecraft may be used to calculate what the direction and distance of the streak should be.

[0099] Once the detected spot and streak data 232 has been extracted, the detected spot and streak data 232 may be used to update locally maintained RSO tracking and spectral information either by updating the orbit parameters for an already tracked RSO or by initializing an orbit for a newly detected RSO. If spectral information is available from another sensor, that spectral information may be used to uniquely identify the RSO, especially if there is a cluster of RSOs nearby that could confuse the observation with the RSO ID or, in the case of a new RSO, add more information about the new RSO. Tracking information, in this context, references the RSO’s or Star’s centroid location and timestamp. When the tracking information is used with ephemeris data from the host spacecraft (position and attitude information), the tracking information is transformed into orbital parameters for RSO’s. Spectral information, in this context,references the instrumental magnitude of the RSO or Star. This information is used to help distinguish between closely clustered RSO’s, as well as convert instrumental to absolute magnitude using calibration Stars.

[0100] As noted, a detected RSO may be an already tracked RSO or a newly detected RSO.

[0101] An already tracked RSO, or an RSO with priori information, may be identified from a catalog of uploaded TLEs. Priori information in this context refers to initial data or assumptions about a satellite’s position, velocity, or trajectory. Priori information may include parameters from previous tracking data, predicted orbital elements, or knowledge of a satellite’s behaviour and mission profile (i.e. , the information needed to know where to look to get an observation). TLE is a standardized data format used to describe the orbit of a satellite around Earth. When the detected RSO has priori information, the orbital parameters associated with the RSO are updated using a process called orbital refinement. Orbital refinement fuses the historical data with the new tracking information to create a more accurate estimate of the RSO’s orbital parameters.

[0102] A newly detected RSO is identified when the position of the detected RSO does not match any previously detected RSO on the on-board catalog (i.e., the detection is not an already tracked RSO). The newly detected RSO is then saved as a new unique RSO ID. The tracking information is used to create orbital parameters for this new RSO ID using a process called Initial Orbital Determination (IOD). Specifically, for a space based optical imager such as optical sensor 202, Angles Only IOD may be used to provide an initial estimate of the orbital parameters. After the RSO ID and initial orbital information have been added to the on-board catalog, subsequent detections are used to refine the orbital parameters using the method described above for an already tracked RSO.

[0103] The RSO tracking and spectral information is stored and used by onboard processing unit 108.

[0104] The detected spot and streak data 232 contains the characterized RSO information determined from the corrected measured image data 230. This information may be used onboard the host spacecraft 106 to update local RSO tracking and spectral information (i.e., onboard spacecraft 106) or may be downloaded to the ground system 112.

[0105] Given that the ground system 112 may receive detected streak and spot data 232 from multiple spacecraft (e.g., multiple instances of spacecraft 106), if there are contemporaneous observations of the same RSO from different spacecraft, then it can be expected that orbit tracking carried out in the enhanced RSO tracking by triangulation module 216 will be substantially improved compared to tracking onboard a single spacecraft (e.g., only on spacecraft 106). This is because triangulation based techniques can yield range information. Additionally, different spacecrafts’ spectral information may be used to characterize attitude and shape.

[0106] In some embodiments, the detected streak and spot data 232 may be used onboard the spacecraft 106 to initiate additional image collections (e.g., via sensor 202 or another payload sensor on board, or by a sensor on board another spacecraft in the same constellation as spacecraft 106 (e.g., via an intersatellite link). This may support more rapid orbit determination and thus enable a more rapid threat assessment to be performed (i.e. , without having to transmit data to ground segment 104 and wait for the ground segment to respond to the course of action). Such an approach may be particularly useful where there is available bandwidth or ground station coverage constraints (e.g., affecting the ability to downlink unprocessed images).

[0107] The detected streak and spot data 232 is provided to the onboard computer 209. The onboard computer 209 formats and packages the detected streak and spot data 232. Formatting includes cropping the area of interest, thereby reducing the volume of data. The formatted and packaged data 232 meets requirements for downlinking data through a transport layer. The onboard computer 209 also adds metadata regarding RSO ID, satellite platform position and attitude, and sensor pointing angle. It should be noted that the accuracy of the satellite platform position and attitude are potential sources of error.

[0108] The data compression module 214 also receives corrected measured RSO data 230.

[0109] The data compression module 214 compresses the corrected measured RSO data 230 according to one or more data compression techniques or algorithms.

[0110] The data compression module 214 reduces the volume of the corrected measured image 230 prior to transmission to the ground system 112 to ensure that the available bandwidth is sufficient for downloading of the measured images 230. In some cases, the data compression module 214 may not be used.

[0111] The output of the data compression module 214 is compressed corrected measured RSO data 234.

[0112] The compressed corrected measured RSO data 234 is provided to the onboard computer 209. The onboard computer 209 formats and packages the compressed corrected measured RSO data 234. The formatted and packaged data 234 meets requirements for downlinking data through a transport layer.

[0113] The detected streak and spot data 232 and the compressed corrected measured RSO data 234 are transmitted to the ground station 112 for further processing, as described below.

[0114] The ground station 112 includes an enhanced streak and spot detection module 236, an enhanced RSO tracking by triangulation module 240, a product generator module 244, and a server-side viewer application 222a.

[0115] The enhanced streak and spot detection module 216 processes the compressed corrected measured RSO data 234 using algorithms or techniques other than those used by the RSO detection and tracking module 212 (i.e. , onboard spacecraft 106) to obtain enhanced detected streak and spot data 238.

[0116] Generally, the computing system in the ground segment comprises significantly enhanced computational capacity compared to the space segment (i.e., onboard the spacecraft). Therefore, the ground segment computing systems can use more computationally demanding algorithms and techniques. The outputs can therefore be expected to more accurately detect and track the RSOs.

[0117] The enhanced detected streak and spot data 236 is provided as input to the enhanced RSO tracking by triangulation module 240.

[0118] The enhanced RSO tracking by triangulation module 240 receives and processes the enhanced streak and spot data 238 and the detected streak and spot data 232 (i.e., from onboard spacecraft 106) along with onboard detected streak and spot data 232 from other spacecraft 106. The purpose of the enhanced tracking mode 240 is to use different satellite viewing geometries to reduce the uncertainty and increase the accuracy in the tracking information. This may be used to improve the accuracy of the derived orbital parameters and reduce the uncertainty ellipsoid. This has many benefits, primarily for SSA, including better knowledge of position for follow up observations. The reduced uncertainty ellipsoid decreases the uncertainty in conjunction analysis. This works by taking the angular position and uncertainty information from sensor in different locationsduring a time period. This position information may then be combined using a data fusion method, such as least squares optimization or Kalman filters. The output of this more refined position accuracy and uncertainty may then be to generate more accurate orbital parameters.

[0119] The enhanced RSO tracking by triangulation module 240 generates enhanced RSO tracking data 242. The enhanced RSO tracking data 242 is provided as input to the product generator module 244.

[0120] The product generator module 244 generates an end user RSO tracking product 246 from the enhanced RSO tracking data 238. The product 246 includes identification and tracking results for RSOs detected or monitored for an end user. The product 246 may be packaged and formatted for consumption by viewer application 222a. The data stream 230, 234 may be archived for subsequent use in improving the RSO detection ML algorithm.

[0121] The end user RSO tracking product 246 is stored in memory of the ground station 112 and used by the viewer application 222a.

[0122] Viewer application 222a is configured to communicate with client-side viewer application 222b executing at user device 128. Viewer applications 222a, 222b may be collectively referred to as viewer application 222. In other embodiments, application 222b may be at a client system (e.g. , client system 132) and not a user device and have multiple feeds from ground and space sensor and open source intelligence (internet, etc.) to predict where to position the spacecraft / sensor for a next observation request.

[0123] Generally, in response to a user input at the user device 128, the viewer application 222b generates a request for an end user RSO tracking product 246 and sends the request to the viewer application 222a at ground station 112.

[0124] The viewer application 222a may retrieve the end user RSO tracking product 246 from memory or other data storage and send the end user RSO tracking product 246 to the viewer application 222b. The viewer application 222b displays the end user RSO tracking product 246 in a graphical user interface.

[0125] In some embodiments, any one or more of enhanced streak and spot detection module 236, enhanced RSO tracking by triangulation module 240, and product generator module 244 may be implemented in the space segment 102 (e.g., at a processing unit onboard spacecraft 106).

[0126] Referring now to Figure 3, shown therein is a method 300 of RSO detection and tracking for space domain awareness, according to an embodiment. The method 300 may be implemented by the system 100 of Figures 1-2.

[0127] One or more steps of method 300 may be encoded as computer-executable instructions which, when executed by one or more processors of a computer device, cause the computer device to perform the method 300 or steps thereof.

[0128] At 302, the method 300 includes collecting optical image data of stars in reference catalogue using an onboard optical sensor (calibration data). The optical sensor may be a repurposed or reconfigured star tracker sensor. The repurposed or reconfigured star tracker sensor may operate differently from a standard star tracking function.

[0129] At 304, the method 300 includes comparing calibration data from 302 against known optical image data in catalogue to obtain corrections data.

[0130] At 306, the method 300 includes correcting optical response of the optical sensor based on corrections data from 304.

[0131] At 308, the method 300 includes collecting optical image data (“RSO data”).

[0132] At 310, the method 300 includes applying corrections to collected RSO data using corrections data from 304 to obtain corrected RSO data.

[0133] At 312, the method 300 includes providing corrected RSO data from 310 as input to RSO detection and tracking module.

[0134] At 314, the method 300 includes analyzing the corrected RSO data using the RSO detection and tracking module to obtain detected streak and spot data.

[0135] At 316, the method 300 includes compressing the corrected RSO data from 310 to reduce volume prior to transmission.

[0136] At 318, the method 300 includes transmitting the compressed corrected RSO data and the detected streak and spot data to a ground system. In some cases, the corrected RSO data may be archived and used to continuously improve the onboard ML RSO detection algorithm.

[0137] At 320, the method 300 includes processing compressed corrected RSO data at ground system using an enhanced detected streak and spot detection module to obtain enhanced streak and spot data.

[0138] At 322, the method 300 includes providing detected streak and spot data and enhanced detected streak and spot data as input to enhanced RSO tracking by triangulation module.

[0139] At 324, the method 300 includes processing detected streak and spot data and enhanced detected streak and spot data by the enhanced RSO tracking by triangulation module to obtain enhanced RSO tracking data.

[0140] At 326, the method 300 includes generating an end user RSO tracking product based on the enhanced RSO tracking data.

[0141] At 328, the method 300 includes transmitting the end user RSO tracking product to an end user device.

[0142] At 330, the method 300 includes displaying the end used RSO tracking product in a GUI at the end user device.

[0143] Referring now to Figure 4, shown therein is a method 400 of determining final orbit parameters, according to an embodiment. The method 400 may be used by any of the systems and methods of the present disclosure, such as system 200 of Figure 2.

[0144] Figure 5 illustrates the orbit refinement of method 400 of Figure 4 in further detail, according to an embodiment.

[0145] Referring first to Figure 4, the method 400 involvestasking an image tracklet 402 of a target RSO and extracting RSO information through the RSO detection and tracking algorithm 212. This outputs a timestamp, pixel location, and magnitude of the RSO in the sensor’s frame of reference (image tracklet data). This may either be done to raw image tracklet data or have been done previously with only the timestamp, pixel location, and magnitude being passed to a coordinate transformation module 404.

[0146] After the image tracklet data has been converted into a universal coordinate frame, such as J2000, by the coordinate transformation module 404, the image tracklet data is then formatted in a time series of information 406 containing timestamp, Right Ascension (RA), Declination (Dec), and magnitude. This information 406 is then used to feed into an Initial Orbital Determination (IOD) algorithm 408.

[0147] Taking range information 410 from different sensors or pseudo range from propagation, the time series range information, in the format of timestamp and range, is then used to calculate a range rate 412 of the object.

[0148] After the range rate information is processed, it is then formatted into a time series of information 414 in the format of timestamp and range rate. This information 414 is then fed into the IOD algorithm 408.

[0149] The IOD algorithm 408 outputs information 416 about the object in a universal coordinate system, including timestamped vectors, position and velocity. This information 416, if the RSO is correlated to a catalog object, is then fed into an orbital refinement module 418. The process executed by the orbital refinement module 418 is described in further detail in Figure 5.

[0150] With a correlated object, priory observations and information 420 may be included to increase the accuracy of the orbit 416 that was determined with the IOD module 408. This additional information 420 may be provided in the format of timestamped vectors, position and velocity, in the same universal coordinate system as information 416.

[0151] After the orbital refinement module 418 is complete and has converged to a suitable answer, final orbital parameters 422 are outputted. The final orbital parameters 422 may be output in the format of timestamped vectors, position and velocity, in a defined universal coordinate system. The final orbital parameters 422 may be output as classical orbital elements with a stated universal coordinate system. The final orbital parameters 422 may be considered an initial or updated orbit suggested by the orbital refinement module 418.

[0152] Referring now to Figure 5, the orbital refinement module 418 is shown in further detail, according to an embodiment. The initial or updated orbit 422 that is being suggested by the orbital refinement module 418 is put into timestamped vectors, position and velocity, format which has the same universal coordinate system as used in the additional observations 420 and information 416 output by the IOD algorithm 408.

[0153] After range and position information 424 is calculated relative to each sensor, the range and position information is then formatted in a timeseries information of timestamp, range, and position (coordinate system depended) 426, to be used to compare the estimated orbit to the measured results at 428 (comparison module 428). This comparison obtains a difference between the estimated and observed position and range.

[0154] The difference between the estimated and observed position and range is converted into a raw score 430. This may be done for the complete time series based ona weighted series of errors between the values to determine how well the estimated orbital solution fits the measured results.

[0155] The score 430 may then be compared to a threshold value. If the score value 430 that is calculated is below the threshold (e.g., 432), the orbital refinement cycle continues. To do this, the orbital estimate 416, that is used in the initial orbit module 408, is updated based on correction values calculated from the errors when comparing the estimated values and the measured values by the comparison module 428. The orbit refinement process may be configured to have a maximum amount of iteration (e.g., number of iteration cycles) that will be run. This is used in case the data is not of sufficient quality to converge on an answer, in which case the system may re-perform orbital refinement when more data on the target RSO becomes available. If the score 430 is above the threshold, the final orbit parameters 422 are output and the orbit refinement process is complete.

[0156] While the above description provides examples of one or more apparatus, methods, or systems, it will be appreciated that other apparatus, methods, or systems may be within the scope of the claims as interpreted by one of skill in the art.

Claims

Claims:

1. A method of space domain awareness, the method comprising:collecting optical image data of a space environment with an optical image sensor onboard a satellite;obtaining resident space object (RSO) data describing an RSO in the optical image data using a processor onboard the satellite, comprising:detecting the RSO in the optical image data; anddetermining tracking information for the RSO.

2. The method of claim 1 , further comprising:performing the collecting optical image data and the obtaining RSO data on a plurality of satellites in a satellite constellation; andcombining the RSO data from the plurality of satellites to obtain enhanced RSO data for the RSO.

3. The method of claim 1, wherein the tracking information comprises data about position and velocity of the RSO over time and is used to predict where the RSO will be in the future.

4. The method of claim 1, wherein the optical image data comprises a tracklet of images, and wherein detecting the RSO in the optical image data comprises:determining image coordinates of the RSO in each image in the tracklet;converting the image coordinates to real world coordinates;performing an initial orbit determination using the real world coordinates, timestamp data, and magnitude data;calculating a pseudo range indicating an altitude the RSO would be at if the initial orbit determination was correct;refining the initial orbit determination using coordinates from other observations; andidentifying refined orbit parameters as final orbit parameters once an expected location of the RSO is sufficiently close to an observed location of the RSO, wherein a determination of whether the expected location and the observed location are “sufficiently close” is determined by the processor according to one or more stored criteria.

5. The method of claim 4, wherein refining the initial orbit determination is performed iteratively until either a maximum number of cycles is achieved ora threshold score has been reached.

6. The method of claim 4, wherein the method is performed with additional tracklets from at least one other satellite providing a different viewing angle.

7. The method of claim 1, further comprising performing enhanced streak and spot detection, comprising:stacking images from a tracklet along with a coordinate list of objects from each image;tracking motion of each object across the stack of images, using Euclidean distance and motion fitting;generating a list of locations for each object;identifying, using motion filtering, the objects that move like stars as stars and the objects that do not move like stars as RSOs;creating an RSO list listing centroid locations of the objects identified as RSO; andcreating a star list listing centroid locations of the objects identified as stars.

8. The method of claim 1 , wherein the RSO data includes detected streaks or spots in the optical image data, the detected streaks or spots representing an angular distance that a detected object traveled over integration time, and wherein the detected streaks or spots are used to distinguish the RSO from starts in the optical image data.

9. The method of claim 1 , wherein detecting the RSO includes distinguishing the RSO from stars in the optical image data using measured streak or spot data.

10. The method of claim 10, wherein distinguishing the RSO from stars includes identifying that the RSO presents as a streak with a different length or orientation from what is expected for a star.

11. The method of claim 1, wherein detecting the RSO in the optical image data includes using a spot or streak detection algorithm to identify and characterize spots or streaks in the optical image data.

12. The method of claim 1, further comprising calibrating an optical response of the optical image sensor, comprising: obtaining observations corrections data using reference data from a reference library; and updating the optical response of the optical image sensor using the corrections data.

13. The method of claim 1 , further comprising updating local RSO tracking information stored onboard the satellite with the RSO tracking information or transmitting the RSO tracking information to a ground system or other satellite.

14. The method of claim 1 , wherein the detected RSO is an RSO that is already being tracked by the processor as existing tracking information for the RSO and the RSO tracking information is used to update the existing tracking information.

15. The method of claim 1, wherein the detected RSO is identified by the processor as a newly detected RSO that is not already being tracked by the processor.

16. The method of claim 1 , wherein the optical image data includes first optical image data collected by a first optical image sensor on a first satellite and second optical image data collected by a second optical image sensor on a second satellite,wherein the first and second optical image data are contemporaneous observations of the RSO from the at least two satellites, and wherein obtaining the RSO data includes performing the detecting and determining on the first and second optical image data on the first and second satellites, respectively, and fusing the tracking information obtained by the first and second satellites.

17. The method of claim 1, further comprising combining the tracking information of the RSO with additional tracking information of the RSO obtained from at least one other satellite to obtain enhanced tracking information for the RSO.

18. The method of claim 17, wherein the combining is performed on the satellite, on the at least one other satellite, or at a ground system.

19. The method of claim 1 , wherein the optical sensor is a star tracker sensor that has been reconfigured to observe RSOs instead of stars.

20. A system configured to implement any of the methods of claims 1-18, the system comprising the optical image sensor and the processor.

21. A system for space domain awareness comprising:an optical image sensor disposed on a satellite, the optical image sensor configured to collect optical image data of a space scene;a processor disposed on the satellite, the processor configured to obtain resident space object (RSO) data describing an RSO in the optical image data, the obtaining comprising:detecting the RSO in the optical image data; anddetermining tracking information for the RSO; anda communication unit configured to transmit or receive the RSO data to or from a system external to the satellite.

22. The system of claim 21 , further comprising:a second optical image sensor disposed on a second satellite, the second optical image sensor configured to collect second optical image data of a space scene;a second processor disposed on the second satellite, the second processor configured to obtain second resident space object (RSO) data describing the RSO in the second optical image data, the obtaining comprising:detecting the RSO in the second optical image data; anddetermining second tracking information for the RSO;a second communication unit configured to transmit or receive the second RSO data to or from a second system external to the second satellite; anda combining module configured to combine the tracking information and the second tracking information to obtain enhanced tracking information for the RSO.

23. The system of claim 22, wherein the combining module is implemented on the satellite or the second satellite, and the communication unit and the second communication unit use an optical intersatellite link.

24. The system of claim 22, wherein the combining module is implemented at a ground system, and the satellite and the second satellite downlink the tracking information and the second tracking information to the ground station via the communication unit and the second communication unit, respectively.

25. The system of claim 21, wherein the optical sensor is a star tracker sensor that has been reconfigured to observe RSOs instead of stars.

Images