Object characteristic determination using brightness measurements

Abstract

A method of determining characteristics of space objects, the method comprising providing multiple space object data points over a period of time; processing the multiple space object data points using an analysis model to create a resident space object type score including the steps of: normalizing the multiple space object data points and removing unneeded data to create normalized data; processing the normalized data using at least one analysis model to create distinct feature data; and utilizing at least one set of distinct feature data set to produce a prediction of at least one characteristic of at least one space object and generating at least one space object type score; and generating a visual output corresponding to the at least one space object type score based on the distinct feature data.

Description

OBJECT CHARACTERISTIC DETERMINATION USING BRIGHTNESS MEASUREMENTSInventors: PACE BALSTER, GAVIN HOFER, GABRIELLE JONES, GHONHEE LEECROSS REFERENCE TO RELATED APPLICATIONS

[0001] This Patent Application is related to and claims priority under 35 U.S.C. § 119(e) to United States Provisional Patent Application Number 63 / 536,348, entitled “OBJECT CHARACTERISTIC DETERMINATION USING BRIGHTNESS MEASUREMENTS,” filed September 1 , 2023, which is incorporated by reference in its entirety for all purposes.FIELD OF THE INVENTION

[0002] The invention relates to using statistical and machine language models for characterizing space objects.BACKGROUND

[0003] The determination of space object characteristics is an increasingly important capability as the space domain becomes more congested and complex. In this regard, the amount of collected Space Domain Awareness (SDA) data has grown considerably over the years, straining existing analysis and processing techniques. Additionally, while an unprecedented amount of data is being collected, the quantity of information gathered on individual objects can vary significantly. This variation necessitates the use of automated and scalable characterization techniques that can adapt to inconsistencies in the density or sparsity of space object data.

[0004] For example, the rapidly evolving space domain, spurred by a dramatic surge in launch rates, the deployment of large satellite constellations, and more complex space operations, has underpinned a need for timely understanding ofspace object characteristics. While global sensor networks are collecting more SDA data than ever, challenges still persist in the ability to fully exploit this information. Human-made space objects exhibit large distances to Earth-based observers, ranging from several hundred to several thousand kilometers. The objects themselves are small relative to those scales (i.e. , on the order of meters). As a result, space objects outside the lowest Low Earth Orbit (LEO) region are not able to be observed in a resolved fashion. Additionally, while some high-priority space objects are tracked near- continuously, the focus often comes at the expense of adequately characterizing other objects. Consequently, when aiming to collect information on a large number of objects routinely, the data available is often non-resolved and sometimes sparse, reflecting the trade-offs made in the prioritization. This necessitates the development of methods that can adequately characterize space objects, even when faced with few nonresolved observations.

[0005] Generally, non-resolved optical observations are composed of astrometric and photometric information. Astrometric information relates to object angular position and rates and can be used in a straightforward manner for orbit determination. However, for further characterization, an object’s reflected sunlight over time offers additional information. In the case of passive observations, a light curve refers to the reflected brightness of an object when arranged in a time series. The brightness is dependent on many factors including object orientation, shape, composition, and solar phase angle. Though analyzing a light curve is an inherently multivariate problem, the encoded information within it offers an opportunity to derive various characteristics from the reflecting body such as size, shape, attitude, mass, and materials. Light curves have been explored in astronomy, and significant research has been done in expanding light curve inversion (LCI). However, LCI often suffersfrom simplifying assumptions, such as a cylindrical shape, or stable, single-axis (or mainly single-axis) rotation. As a result, only a few combined shape and attitude approaches have been explored.

[0006] Because of these shortcomings, shape-surface determination has been approached using filtering approaches. To provide the necessary initial conditions and to limit the vast parameter space to allow for convergence, the problem has been made tractable by using simple a priori shapes such as cubes, and cylinders and obtaining a weighting in a mixture of experts, filter banks or multiple hypotheses techniques. However, the addition of orbital information to narrow down ambiguities via the use of orbital perturbations has not yet shown significant improvements.

[0007] Due to these challenges, LCI methods often require making limiting assumptions that reduce their generality to a subset of objects, such as convex objects with uniform material. These assumptions do not adequately address the need for better characterization of human-made objects across different sizes, shapes, and orbital regimes. While the full inversion problem for shape and attitude is extremely difficult, methods for partial inversion, sometimes referred to herein as “Blob Identification” (or Blob ID) aim at partial inversion results. With Blob ID, a full inversion is not sought, but rather extracting certain features of the object, often in a trade-space analysis against multiple plausible options, similar to a mixture of expert approaches is used. This greatly diminishes the assumptions that need to be made, while providing a focus on the more readily observable characterization features.

[0008] Furthermore, Machine learning (ML) methods have shown applicability across a variety of domains, including the characterization of space objects. ML can be subdivided into classical and deep learning methods. Classical methods, as defined here, are a set of algorithms and statistical models that enablecomputers to learn relationships between features in a data set to make predictions on new data. Classical methods often focus on interpretability and use structured datasets. Specific classical methods include Logistic Regression, Decision Trees, Random Forests, and Support Vector Machines.

[0009] These approaches are particularly advantageous when the training data set is relatively small and lacks complexity. New advancements in machine learning, including deep learning, have shown promising results across many domains including computer vision and natural language processing. Deep learning methods typically refer to multi-layered artificial neural networks that comprise hidden layers of interconnected nodes to expose complex feature relationships by learning on a large training data set. In general, both classical and deep learning ML methods have potential due to their ability to handle non-linearities, high dimensional data, and their tolerance to noise. Additionally, ML models can be highly efficient to run after they have been trained. These traits present applicability for the multi-variate object characterization problem discussed above.

[0010] ML approaches have been previously investigated to assist in extracting shape and attitude information from the inversion of light curves. For example, a classical model, Random Forest model, performed just as well as a deep learning approach, specifically a Long Short Term Memory (LSTM) model for predicting the operational status of resident space objects (RSOs) in LEO. The term “track” is defined as all the time-series measurements for a single observed pass overhead of a given satellite.

[0011] Other efforts have been made to apply supervised and unsupervised machine learning techniques to partially solve the LCI problem. For example, deep learning algorithms to learn the inverse relationship between lightcurves and shape have been attempted. Using a supervised approach, they were able to classify different shapes of rocket bodies using a Convolutional Neural Network (CNN). Methods for unsupervised learning like clustering and dimensional reduction have been investigated for satellite classification but have not been conclusive on their own.

[0012] Thus, systems and methods that provide improvements in the ability to predict space object type using a model that is scalable to a variety and increasing number of non-resolved observations, even when the observations are relatively few and are collected over short time durations, are desirable. SUMMARY

[0013] The present disclosure provides a computer implemented method of determining characteristics of space objects from multiple space object data points over a period of time, In accordance with various aspects of the present disclosure, the method comprises the steps of processing the multiple space object data points using an analysis model to create a resident space object type score including the steps of:• normalizing the multiple space object data points and removing unneeded data to create normalized data;• processing the normalized data using at least one analysis model to create distinct feature data; and• utilizing at least one set of distinct feature data set to produce a prediction of at least one characteristic of at least one space object and generating at least one space object type score; and• generating a visual output corresponding to the at least one space object type score based on the distinct feature data.

[0014] In accordance with various aspects of the present disclosure, the analysis model comprises at least one of a distinct set of statistical models. In accordance with various aspects, the analysis model comprises a machine learning model and in some embodiments, the machine learning model is a convolutional neural network.

[0015] In accordance with various aspects of the present disclosure, the space object data is ground-based and / or space-based. In accordance with various aspects of the present disclosure, the space object data further comprises space object meta-data.

[0016] In accordance with various aspects of the present disclosure, the computer implemented method may further comprise the step of providing an alert associated with a change in one or more photometric parameters.

[0017] In accordance with various aspects of the present disclosure, the computer implemented method may further comprise the step of receiving photometric data from a plurality of ground-based and / or space-based sensors.

[0018] In accordance with various aspects of the present disclosure, the computer implemented method may further comprise automated data processing, and in accordance with some aspects, the automated data processing further comprises generating an event probability output and / or generating an object type probability output.

[0019] In accordance with various aspects of the present disclosure, the analysis model processes the normalized data using at least one of:• an Orbital Element Family ID model comprising an Orbital Element statistical model for building a distribution of object families based on orbital element information to output an In-Family object type score;a Photometric Family ID model comprising a Photometric Family statistical model for building a distribution of object families based on photometric information to output an In-Family object type score;• an ML Target Classifier model comprising a machine learning model to output an In-Family object type score based on space object data;• a Mission Type Family ID model comprising a Mission Type statistical model based on a normalized magnitude and a solar phase angle of a payload mission type to output an In-Family object type score; and• an Area to Mass Ratio Family ID model for building a distribution of object families using a solar radiation pressure coefficient to output an In-Family object type score.

[0020] In accordance with various aspects of the present disclosure, the Mission Type statistical model is a multi-dimensional statistical model.

[0021] In accordance with various aspects of the present disclosure, the computer implemented method may further comprise Class Violation Indicators, and the Class Violation Indicator may reference an analysis model to determine whether a specific object falls within a distribution for set object types and compares the output to a public catalog designation, and wherein if the distribution for set object types and the public catalog designation are different, then outputting that the space object violates a class based on the analysis model.

[0022] In accordance with various aspects of the present disclosure, the computer implemented method may further comprise analyzing the Class Violation Indicators to generate a Target Nomination application. In accordance with some aspects, the Target Nomination application may combine the output of a plurality of Class Violation Indicators and compares a deviation of the space object from theplurality of Class Violation Indicators, then outputting a target nomination priority ranking based on a degree to which objects are violating its class.BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The accompanying drawings provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention, and together with the description serve to explain the principles of the invention.

[0024] FIGURE 1 illustrates a computer implemented method of determining the characteristics of space objects in accordance with the present disclosure;

[0025] FIGURE 2 illustrates a ML Target Classifier approach using a Light Gradient Boosting Machine model and multiple decision trees in accordance with the present disclosure;

[0026] FIGURE 3 illustrates the Celestrack SATCAT operational status used to define active and inactive payload labels in accordance with the present disclosure;

[0027] FIGURE 4 illustrates the architecture of the Aggregate Random Forest Classifier in accordance with the present disclosure;

[0028] FIGURE 5 illustrates how results from Blob ID are used as input features for Random Forest to predict the object classes in accordance with the present disclosure;

[0029] FIGURE 6 illustrates an architecture of the BloblD-Aggregate Random Forest classifier in accordance with the present disclosure;

[0030] FIGURE 7, 8 and 9 illustrate how two classifiers that use statistical aggregate quantities as input features exhibit more favorable precision / recall andsensitivity / specificity tradeoffs compared to the BloblD-only classifier in accordance with the present disclosure;

[0031] FIGURE 10 illustrates confusion matrices for three classifier methods in accordance with the present disclosure;

[0032] FIGURE 11 illustrates the importance of statistical aggregate features used in Method 1 in accordance with the present disclosure;

[0033] FIGURE 12 illustrates the importance of the features derived from Blob Identification used in Method 2 in accordance with the present disclosure;

[0034] FIGURE 13 illustrates the importance of combined statistical aggregate and Blob Identification features used in Method 3 in accordance with the present disclosure; and

[0035] FIGURE 14 illustrates the workflow for post-maneuver UCT resolution correlating UCTs to objects with fewer observations compared to traditional methods in accordance with the present disclosure. DETAILED DESCRIPTION

[0036] Persons skilled in the art will readily appreciate that various aspects of the present invention may be realized by any number of methods and apparatuses configured to perform the intended functions. Stated differently, other methods and apparatuses may be incorporated herein to perform the intended functions. It should also be noted that the accompanying drawing figures referred to herein are not all drawn to scale but may be exaggerated to illustrate various aspects of the present invention, and in that regard, the drawing figures should not be construed as limiting. Finally, although the present invention may be described in connection with various principles and beliefs, the present invention should not be bound by theory.

[0037] The above being noted and as will be described in more detail below, the present disclosure contemplates devices and systems that provide improvements in the ability to predict space object type. In accordance with various aspects of the present disclosure, models that are scalable to a variety and ever- increasing number of non-resolved observations, even when the observations are relatively few and are collected over short time durations, is provided.

[0038] For example, in accordance with the present disclosure, computer implemented methods and systems comprise a series of independent, statistical and ML models that each process different unresolved observation data fields and test a unique hypothesis about a space object using that data. The results from each of those models about an object are then modified and combined in different ways to provide actionable space domain awareness (SDA) intelligence to the user of the system.

[0039] For example, in accordance with various aspects of the present disclosure, aggregate statistical features using various optical observations as inputs may be contrasted with features generated from a partial LCI approach. The methods are compared independently and in conjunction with each other, with the combined aggregate and partial inversion feature set shown to offer improvement over either, in isolation, enhancing the tools available to the SDA community for object characterization, thereby improving understanding of space objects to better inform decision-making in an increasingly dynamic space environment.

[0040] The above being noted, in accordance with various aspects of the present disclosure, in general and with reference to FIG. 1 , a computer implemented method of determining the characteristics of space objects, comprises first providing multiple space object data points over a period of time and processing the multiplespace object data points using an analysis model to create a RSO object type score which is a probability value of that object being "in family" with a set of object types. The processing of the multiple space object data points may include, among other steps, normalizing the multiple space object data points and removing unneeded data to create “normalized data” (as described below) and then analyzing the normalized data using at least one statistical model. In this regard, normalizing data isolates changes in the relevant features to actual satellite behavior but not necessarily the range or specific solar phase angles.

[0041] Next, the computer implemented method generates a visual output corresponding to the RSO object type score based on the distinct feature data. Data Analysis

[0042] The above being noted, in accordance with various aspects of the disclosure, the above pre-processed data is provided to various models for analysis. For example, the models and the data fields processed, may comprise, among others, alone or in combination:• Orbital Elements Family ID including information related to the semimajor axis, eccentricity, and inclination of the RSO for building a distribution of object families based on orbital element information to output an In-Family object type score.• Photometric Family ID including information related to the magnitude and solar phase angle of the RSO, for building a distribution of object families based on photometric information (or parameters) to output an In-Family object type score. In some instance, an alert associated with a change in one or more photometric parameters may be provided.ML Target Classifier model including space object data of the RSO,using a machine learning model to output an In-Family object type score based on space object data.• Mission Type Family ID including information related to a normalized magnitude and a solar phase angle of the RSO to output an In-Family object type score. In accordance with some aspects, the Mission Type Family ID model may comprise a two-dimensional statistical model.• Area to Mass Ratio Family ID including information related to the solar radiation pressure coefficient of the RSO, for building a distribution of object families using the solar radiation pressure coefficient to output an In-Family object type score.

[0043] In accordance with the present disclosure, the models output a result corresponding to an In-Family object type score. In this regard, each RSO is given an In-Family object type score from the original data source (e.g., active payload, rocket body, debris, etc.). Next, the relevant data fields from the processed data are used to create distributions of distinct object types. For example, the solar radiation pressure values for each RSO labeled as object type “Rocket Body” will be grouped together into a distribution. Next, an In-Family Score that ranges from 0 to 1 is calculated for each value in the object type distribution to quantify it’s conformity. For example, if Object B is given an In-Family score of 0.3 with Rocket Bodies based on its solar radiation pressure coefficient, but its In-Family Score with Active Payloads is 0.8, then this means that based on this one Area to Mass Ratio model, Object B is more in family with Active Payloads than Rocket Bodies. This type of analysis may be done for every RSO across all of the statistical models above, as well as others now known or as yet unknown.

[0044] In accordance with the present disclosure, the ML TargetClassifier may take a modified approach to the processing. For example, in accordance with the present disclosure, the ML Target Classifier first receives relevant time series electro-optical observations from the data source being used. The data is segmented into tracks, which are sequences of electro-optical observations with the following variables:• Observation time;• Apparent magnitude;• Solar phase angle;• Right ascension; and• Declination.

[0045] Next, the following aggregate quantities for each channel over each track are calculated:• Mean;• Maximum;• Minimum; and• Variance.

[0046] The aggregate quantities are then processed by the ML Target Classifier model and the output therefrom contains a probability (i.e., a percentage) that the object is a particular object type.

[0047] With reference to FIG. 2, the ML Target Classifier approach may use a Light Gradient Boosting Machine model, which uses multiple decision trees trained on bootstrapped subsets of the training data. The final prediction is the majority vote of class probabilities averaged over all tree predictions. In accordance with this embodiment, the ML Target Classifier is training on 80% of the data and then testing on the remaining 20%.

[0048] Systems and methods in accordance with the present disclosure are advantageous in that, by comparing an object to a distribution of similar objects, the object can be characterized to determine indications of object characteristics without prior observations of that particular object.Results Visualization

[0049] In accordance with various aspects of the disclosure, the combined results from each statistical model and the ML Target Classifier, provides a list of results that help determine whether an object is acting like the object type it appears to be, for example, based on known cataloged examples. To provide an output that is useful to the user of the system, the results are processed to provide a visualization in a user-friendly manner.

[0050] In this regard, the computer implemented method of determining the characteristics of space objects may comprise Class Violation Indicators. The Class Violation Indicators output whether a specific object falls within the distribution for set object types or not and compares the output to a public catalog designation. If the distribution for set object types and the public catalog designation are different, then an output that the specific object violates a class based on that specific model is given. In various embodiments, the Class Violation Indicators may be analyzed to generate a Target Nomination.

[0051] For example, for each statistical model, a threshold for what is considered out-of-family is determined. By way of example, an In Family score of less than 0.7 for Rocket Bodies using the Area to Mass Ratio ID model is considered out- of-family. If an object is out-of-family for the object type it appears to be, then a TRUE flag for that model is designated, and if the object is In Family, a FALSE flag for that model is designated. This provides a simple output to the user to show whether anobject is showing any signs of uncharacteristic, or out-of-family, behavior.

[0052] In accordance with various aspects of the disclosure, the overall results are processed for a specific object across all models to determine whether an object is showing signs of uncharacteristic behavior. For example, if the object is out- of-family across many models, it may be designated with a priority number and highlighted as a potential target. For instance, if an object is out-of-family across 2 of the 4 models and the ML Target Classifier also classifies it as different than it appears to be, then the object is highlighted as a Priority 2 and added to the Target Nomination list.

[0053] The above being noted, various examples of systems and methods in accordance with various aspects of the disclosure follow.Data Processing

[0054] Brightness, expressed as calibrated apparent magnitude, depends on several factors. These include the satellite’s shape, albedo, material composition, attitude, the angle between the observation vector and the sun vector (solar phase angle), and the distance from the observer and object to the sun. Given that brightness fluctuates based on these factors, the change in brightness over time, or the light curve, can be used to further characterize the object. When the light curve is used in conjunction with the astrometric quantities, an even more comprehensive characterization of the object can be attained.

[0055] Additionally, sequences of electro-optical observations (data) on a given object can be analyzed to predict the object’s class (e.g., active payload, inactive payload, debris, or rocket body). Because both astrometric and photometric data are used for input features to the ML Target Classifier model, a track, as defined here, is made up of observations containing both astrometric and photometricinformation. In this respect, tracks varying from 10 to 15 observations may be used, with durations on the order of several seconds. The tracks analyzed are produced by objects ranging from Medium Earth Orbit (MEO) to Geosynchronous Orbit (GEO), with most tracks being from GEO objects. In accordance with various aspects of the present disclosure, values from each observation which may be used for analysis include time in seconds from the first observation in the track; brightness, expressed as calibrated apparent magnitude; right ascension and declination in degrees; right ascension and declination rate-of-change in degrees per second.

[0056] Thus, in accordance with various aspects of the disclosure, the multiple space object data may be, for example, photometric data and may be either or both of ground-based and space-based. Space object data may further include a variety of space object meta-data. Space object meta-data includes a plurality of data such as processed data from initial orbit determination, track association, object correlation, photometry and astrometry processing including the magnitude from the observations as well as positional and orbital information that's associated with the observations. The space object data may come from any now known, or as yet unknown, source that contains unresolved observation data over time. The space object data is pre-processed to remove data that is an outlier, unneeded or otherwise harmful. The space object data is normalized so all sources are in a similar and useable format. Next, any required fields that are not present in the raw space object data that the processing models require are calculated. Additionally, in accordance with various aspects, the system may continuously receive new space object data as it becomes available from the various sources.Labeling the Data

[0057] In accordance with various aspects of the present disclosure,data sources, for example, an open-source website that maintains a satellite catalog (SATCAT) known as “Celestrack” provides information about RSO orbital details, owner / operator information, object tags, and current operational status. From the SATCAT, object tags (e.g., payload active, payload inactive, debris, and rocket body) are collected for all NORAD IDs in the catalog. The NORAD IDs with the corresponding tags are mapped to a ground-truth data set to enable evaluation of the supervised ML classification approaches. FIG. 3 illustrates the Celestrack SATCAT operational status used to define active and inactive payload labels.Classifier Training and Validation

[0058] In accordance with various aspects of the present disclosure, verification and validation for quantifying the robustness and generalization of ML models to new cases and data are important. To test this, it is typical to maintain a hold-out set of data that the model has not used or seen before to test the performance of the model when introduced to new cases. For example, a method referred to as Repeated Stratified K-fold Cross-Validation may be used to evaluate each classification approach. Repeated Stratified K-fold Cross-Validation is a technique that provides a robust measure of model performance by partitioning the data into ’k’ randomized subsets or ’folds’ multiple times while ensuring the class distribution of each fold mirrors the overall distribution. For each testing cycle, the classifier is trained and validated ‘k’ times, each time using a different fold as the validation set (also known as a ’’holdout” set), and the remaining folds as the training set.

[0059] To increase the reliability of the performance estimates for the model, multiple testing cycles are carried out with randomized fold selections in each cycle. For each of these testing cycles a new randomized subset with evenly distributed class labels is selected prior to splitting into folds. Additionally, each fold is stratified to inherit the balanced class label distribution. Balanced subsets are usedboth to address difficulties in training predictive models on unbalanced datasets, and to improve the interpretability of the performance results. For performance evaluation, the mean of each of the performance metrics is calculated across all folds and testing cycles. All performance results in this study are produced using 5 folds, repeated across 100 test cycles, for a total of 500 individual classifier fits.

[0060] Notably, the folds are produced by selecting tracks by catalog ID, so that each individual RSO is isolated to one fold for cross-validation. This approach is done to prevent a form of data leakage that could occur if the model is evaluated using tracks produced by any of the same objects found in the training set.Aggregate Random Forest Classifier

[0061] The Random Forest model may be used due to its comparable performance to deep learning methods for object type prediction. Additionally, selecting a relatively predictable model, such as Random Forest, allowed for the isolation of the predictive power of different input features from that of the aggregate statistical features.

[0062] T o transform time-series track data into a set of interpretable input features for the Random Forest classifier, the following statistical aggregate quantities are computed for all six channels over each track: mean, maximum, minimum, and variance of the raw values as well as the back-differences of the raw values. Altogether, there are 48 aggregate features available as inputs to the Random Forest. The architecture of the Aggregate Random Forest Classifier is shown in FIG. 4.

[0063] These aggregate features may be used instead of the raw values from the observations such that tracks of varying lengths and time intervals can be used, and to reduce sensitivity to random fluctuations.Blob Identification

[0064] In accordance with various aspects of the present disclosure, in lieu of a full inversion, the equations governing the reflected light can also be used to ascertain more generalized, incomplete characterization information, supporting the association of objects in different categories. Such information, often dubbed Blob Identification (Blob ID), can stand on its own or be used, as in the case of this paper, in conjunction with machine learning methods.

[0065] The Blob Identification method relies on the principle of finding relations that fit a certain set of generalized albedo shape and frequency relations that are in agreement with the proposed satellite categories. Based on the fit, which often is poor, weights can be calculated, similar to a mixture of expert fashion. A difference is that no full-scale models are fit, but sub-facets in proportionality relations rather than absolute models. Furthermore, no full attitude profiles are matched, but only frequencies are extracted.

[0066] The principal relations used are the reflection properties of a flat plate and round surface reflection. The irradiance Ipiate from a flat plate defined by its surface normal is captured by the following proportionality relation:I oc cos0s(p cos0o +q-6(0s -6 )6(cps ~(po +77)) (1)

[0067] where 0o is the out-of-plane angle between the surface normal and the observer direction, and 0s to the sun direction, respectively, (ps and (po are the corresponding in-plane angles, guaranteeing the glint condition. As is known, the sun is an extended source and there is a 0.5-degree deviancy in the glint direction allowed, however, solar panels might also be slightly misaligned making this difference negligible, p and q are the albedo-area relations of Lambertian to specular reflectionproperties. For Blob ID, the relation between p and q is not fixed. Additionally, the condition that they have to add up to 1 is not enforced, hence acting as a general scaling parameter accounting for the entirely coupled area-albedo relation.

[0068] With the flat-plate proportionality relation, any box-bus configuration and solar-panel (single or dual wing) relation can be modeled for a fit to an actual light curve. For a controlled box-wing satellite, the solar panels are oriented towards the sun, so one flat plate orientation has a necessary orientation of angle between the normal and the sun 0S = 0 and, hence, the angle between the normal and the observers is the same as the overall phase angle 00 =a. For the reflectivity, it is irrelevant if it is a two or one solar panel configuration. The eventual blockage of the view to a panel to the observer by the bus is equivalent to a different albedo-area and hence part of the scaling in the fit. Maximally three surfaces of a box-shaped bus are visible, constrained in a relative 90-degree configuration. Only two sides at most are illuminated for a controlled satellite with an approximately 90-degree angle to the actuating panels, which are sun-oriented. Side 1 , which is just offset by the solar-panel rotation angle / 3 to follow the sun, leads to a 0s = / 3 and O =a -fi.

[0069] A satellite can go through several instances of rotational and tumbling motions when no longer attitude stabilized and fewer restrictions can be applied, as the orientation towards the sun is lost for any surface. A constraint that can be enforced at any time is the one of the overall phase angle a =0s +0o. This constraint can be enforced for a cube configuration representing the box, where at most three sides are visible at any time. In this regard, the solar panels are assumed to be at a random actuation angle and in a 90-degree configuration relative to the box.

[0070] The stabilized and non-stabilized box-wing satellites are most distinct in their relative synodic variation that is observable in the light curve,corresponding to the different attitude profiles. For the analysis, the periodogram method is far superior in terms of robustness in the presence of noise over a Fourier analysis. Box-wing satellites in terms of their relative inertia distribution and relative rigidity of the object itself, tend to result in synodic periodic patterns, which repeat over short time periods. As such, an additional constraint is applied in terms of detecting distinct periods over-weighing a non-stabilized hypothesis relative to a stable box-wing one.

[0071] Additionally, a large group of objects are cylindrical in shape. This includes spin-stabilized objects and their debris following end-of-mission and the larger group of upper stages, some of which also experience a spin-up during deployment. Spin-stabilized satellites following end-of-life will continue to maintain spin through momentum conservation. Although eddy currents contribute to a slowing of the spin, collected torques create rotations around other axes. Recently passivated spin-stabilized objects are indistinguishable from still spin-stabilized ones over short observation times in the light curves.

[0072] Cylindrical objects can be characterized via the reflection off the mantle, which is orthogonal to the spin axis identical to the spherical reflection, leading to the following irradiance relation: / sphere <x cosQo' p(sin(a)+( r -or) cos (or) (2)

[0073] where o is the angle of the cylindrical axis alignment relative to the observer direction, and a is the phase angle. The top and bottom face, which are flat plates, are governed by Equation 1 above. Nozzles are neglected, as they are absorbed into the mantle albedo-area-relations.

[0074] For a cylindrical spin-stabilized object, the spin axis alignsorthogonal to the sun, bringing Equation 2 above to its full bearing. The angle to the observereo is then determined via the orbital plane and can be computed directly from the state of the object at each point in time during the light curve. A cylindrical uncontrolled object does not maintain this relation. Therefore, the combined proportionality relation of Equations 1 and 2 are combined with an open relation of the angleeo, enforcing the orthogonality of the mantle relative to its flat top and bottom.

[0075] For a cylindrical uncontrolled object, significant attitude motion may take place, which clearly discriminates it from a spin-controlled object of the same shape. The light curves of spin-stabilized cylindrical objects are generally very flat, due to the gentle cylinder mantle, which can be detected using the periodogram analysis. Freely tumbling cylinders fit the curved surface relation, Equation 2, while also showing significant periods even over short time spans. It has to be noted that there generally exists non-uniform distribution of mass, e.g., upper stages with slosh and objects with a heavy motor on one end and a (mostly) empty hull at the other. As such, the attitude motion is often more complex and has more than one dominant period present. Again, the periodogram method can be used to over- or underweight hypotheses that stem from the intensity proportionality fits.

[0076] To predict object classes using the Blob Identification method, first, the irradiance of the observation is computed: / = / 0(10)-0.4m (3)

[0077] where Io is the reference irradiance, and m is the calibrated apparent magnitude. Note that since absolute scaling is not required for Blob Identification, Io can be set to 1.0 while preserving the relative change of irradiance over time.

[0078] In accordance with the present disclosure, four different modelsbased on the relationships discussed above are fitted to the light curve to minimize the mean squared error (MSE) of irradiance. The four models represent stabilized and non-stabilized versions of both a boxwing satellite and a cylinder. The final MSEs of a batch-optimized fit are then used along with Boolean flags for whether or not each model converged as features in a random forest classifier to predict the object class. The Random Forest classifier is used to learn the correlation of these features to object class from the data and get an estimate of the full predictive power of these residuals. FIG. 5 shows how the results from Blob ID are used as input features for Random Forest to predict the object classes.Blob ID-Aggregate Random Forest Classifier

[0079] In accordance with the present disclosure, a final classification method, a BloblD-Aggregate Random Forest Classifier, is a combination of the previous two methods above. This method uses all of the aggregate quantities computed for each track discussed above, in addition to the normalized mean squared errors and Boolean convergence flags of the Blob Identification models, as features for a random forest classifier. With the forty-eight statistical aggregate quantities, four MSEs, and four convergence flags, the Random Forest in this approach takes in a total of fifty-six input features. The architecture of the BloblD-Aggregate Random Forest classifier is shown in FIG. 6.Performance Metrics

[0080] In accordance with the present disclosure, the three classification methods described above can be evaluated using the performance metrics in Table 1 below. These metrics can be used to evaluate both the performance in distinguishing individual classes from all the others (one vs. all), as well as combined performance. Additionally, macro averaging, a method that finds the overall metric by averaging themetric over all class types without considering class imbalances, is used to determine the combined precision, recall, and f1 score.Table 1 : Machine Learning Performance MetricsRESULTS

[0081] As described above, repeated stratified k-fold cross-validation can be used to evaluate the performance of each classification method. For example, Table 2 below illustrates a comparison of the three classification methods explored, in terms of balanced accuracy for individual classes (one vs. all), as well as overall accuracy. Balanced accuracy is selected over regular accuracy for individual class predictions because the regular accuracy would be artificially inflated due to the class imbalance in the one versus all classification problem. For the overall accuracy, balanced and regular accuracy are generally equivalent because a balanced subset is used for evaluation. Note that for the balanced accuracy on individual classes the baseline corresponding to no predictive power is 50%, and for the overall accuracy on four classes the baseline is 25%.Table 2: Individual Class Balanced Accuracy and Overall Accuracy for each Classification Method

[0082] As shown in Table 2, the Blob Identification classifier performs best when distinguishing active and inactive payloads from other classes, while the classifiers that use statistical aggregate quantities perform better on debris and rocket bodies. Note that the Blob Identification method is generally more effective on longer tracks, as it relies on phase angle dependencies and periodic temporal dynamics. In accordance with the present disclosure, tracks of less than 10 seconds in duration and phase angles spanning less than 150 arcseconds were used. Therefore, it is noteworthy that the Blob Identification method still demonstrates some predictive power when applied to these tracks. When combined with the statistical aggregate features, Blob Identification only provides a marginal performance increase, suggesting that the useful information provided by Blob Identification for predicting object class mostly overlaps with that provided by the statistical aggregate features. To ensure that the increase in performance observed with the addition of Blob Identification is not due to random variation, the repeated k-fold cross validation process is repeated 10 times. The standard deviation over these 10 runs in macroaveraged balanced accuracy is 5.45x10“3%. The standard deviation of the overall accuracy over these 10 runs is 1.59x10“2%.Classification Reports

[0083] Tables 3, 4, and 5 herein illustrate the performance of the three classification methods described above. The selected performance metrics are precision, recall, f1 score, and balanced accuracy, as well as overall accuracy for multi-class performance. For precision, recall, f1 score, and overall accuracy, the baseline (corresponding to no predictive power) is 0.25. For balanced accuracy, the baseline is 0.5.Table 3: Aggregate Random Forest classifier performanceTable 4: BloblD-Only Random Forest classifier performanceTable 5: BloblD-Aggregate Random Forest classifier performance

[0084] As seen in Tables 3, 4, and 5, both classification methods that incorporate the statistical aggregate quantities significantly outperform the method that only uses the outputs from Blob Identification as input features. Including the outputs from Blob Identification results in a marginal increase across all measured performance metrics indicating some unique predictive power in the method.Precision-Recall and Receiver Operating Characteristic Curves

[0085] In accordance with the present disclosure, Precision-Recall (PR) and Receiver Operating Characteristic (ROC) curves are both generated by defining a set of varying classification thresholds from 0 to 1. A prediction is flagged as a specific class if the classifier output probability for that class exceeds the classification threshold. For each threshold, classification results are re-evaluated in order to depict trade-offs between different metrics. PR curves represent the one-vs-all precision and recall for each class across the range of classification thresholds. ROC curves represent the one-vs-all true positive rates and false positive rates for each class across the range of classification thresholds. Depending on the use case of a given classifier model, the PR and ROC curves can provide valuable insight into the effectsof setting a desired level of performance in one metric, such as the one-vs-all true positive rate for a specific class. FIGS. 7, 8, and 9 show the PR and ROC curves for each classification method. For example, FIGS. 7, 8, and 9, show the two classifiers that use statistical aggregate quantities as input features exhibit more favorable precision / recall and sensitivity / specificity tradeoffs compared to the BloblD-only classifier.Confusion Matrices

[0086] In accordance with the present disclosure, a confusion matrix is a grid of probabilities that represents the percentages of correct and incorrect classifications for each class. This can yield valuable insight into the specifics of misclassifications in a given model. The values shown in this section are normalized against the actual number of samples in the class, so that the main diagonals are equivalent to the recall for each class. FIG. 10 shows the confusion matrices for the three classifier methods.

[0087] For example, FIG. 10 shows that the BloblD-Aggregate classifier demonstrates about 1 .20% higher recall for the active payload class when compared to the Aggregate classifier. This result indicates that the Blob Identification outputs help the Random Forest algorithm resolve edge cases in situations where inactive and active payloads are difficult to distinguish. For both of the classifiers fit using the aggregates of the raw data, the portion of the confusion matrices relating inactive payloads, debris, and rocket body classifications are identical to 2 decimal places, showing that the preprocessing does not provide as much aid in distinguishing these classes.

[0088] The confusion matrix for the BloblD-Only classifier provides insight into which classes the Blob Identification methods may be most useful fordistinguishing class types. The classifier is the most accurate on the active payload class, with a recall of 40.7%. This is an intuitive result because the Blob Identification methods are centered around determining whether or not an object is stabilized, and the active payload is the only class corresponding to stabilized objects. Additionally, the BloblD-Only classifier is able to distinguish debris from inactive payloads relatively well. However, the recall on rocket bodies is 26.3%, which is only marginally more accurate than random guessing. This method performs poorly when distinguishing between rocket bodies and debris, suggesting that the Blob ID methods output similar results for these two classes.Feature Importance

[0089] In accordance with the present disclosure, feature importance is calculated as the mean decrease in impurity for each feature in the Random Forest model. Feature importance depicts how much emphasis the Random Forest places on a feature for differentiating between the different classes. A feature importance of zero means that the feature is not used at all in the Random Forest. FIG. 11 shows the importances of statistical aggregate features used in Method 1 .

[0090] As seen in FIG. 11 , the most important statistical features for class differentiation are the mean, minimum, maximum, and variance of magnitude, and mean, minimum, and maximum of right ascension rate. The high importance associated with right ascension rate is suspected to be due to it being a proxy for orbital altitude. Several features exhibited little to no importance, including the variance of the back-differences of solar phase angle, right ascension, and declination, as well as all features derived from the back-differences of right ascension rate and declination rate. These features could likely be removed without degrading the performance of the classifier.

[0091] FIG. 12 shows the importances of the features derived from Blob Identification used in Method 2. From FIG. 12, it can be seen that the mean squared errors of all four Blob Identification models contribute roughly equally to the class predictions. The convergence flags, on the other hand, contribute much less to the final predictions, with two of these features exhibiting zero importance.

[0092] FIG. 13 shows importances of combined statistical aggregate and Blob Identification features used in Method 3. As seen in FIG. 13, the addition of the Blob Identification features did not significantly impact any of the importances of the statistical aggregate features. While the overall importance of the Blob Identification decreased significantly, showing informational overlap with the statistical aggregates, they remained similar to one another. These results further support that most, but not all, of the information provided by Blob Identification for class prediction is captured by the statistical aggregate features. Furthermore, it shows that the Blob Identification features are not highly correlated to any individual statistical aggregate feature or simple combination of such. Rather, the informational overlap comes from deeper representations of these statistical aggregates that the Random Forest is able to express.Conclusions

[0093] Thus, in accordance with various aspects of the present disclosure, systems and methods contemplated herein demonstrate that object classes, delineated by active payloads, inactive payloads, debris, and rocket bodies, can be predicted from short tracks, composed of 10 to 15 electro-optical observations, with durations of less than 10 seconds.UCT (Uncorrelated Track) Processing

[0094] In accordance with various aspects of the present disclosure, acombination of kinematic correlation algorithms with non-kinematic statistical algorithms to correlate and identify UCTs is used. In this regard, historically, UCTs are correlated solely using kinematic correlation algorithms, which work by propagating the orbital dynamics of catalog objects and comparing the propagated states with observations. This is very data and calculation intensive and can fail when there are large maneuvers that cause the UCT to fall outside the set boundaries.

[0095] In accordance with various aspects of the present disclosure, kinematic correlation with non-kinematic correlation using statistical models for both candidate filtering and to increase the confidence of correlations are possible. In this regard, the workflow incorporates statistical distributions of observable quantities generated from historical observations on objects in the catalog, which can be compared to the corresponding values in the UCT observations to adjust the confidence of the correlation. Stated another way, systems and methods in accordance with various aspects of the present disclosure, look at other observations about the objects, like their photometries, to help correlate objects with UCTs.

[0096] In accordance with various aspects of the present disclosure, the process has five primary steps:1. Stochastic Initial Orbit Determination (SIOD) creates a probability distribution for the object’s state based on the UCT observations.2. Statistical Object Identification compares observable non-kinematic quantities of the UCT to distributions stored in a database for filtering and improved confidence determination downstream.3. Candidate Filtering filters UCT-object candidate pairs based on both kinematic and non-kinematic quantities from the SIOD and Statistical Object Identification steps.4. Probabilistic Kinematic Track Correlation finds the feasibility of correlation between the UCT and catalog object by propagating the probability density function (PDF) of the object forward and of the UCT backwards to find the maximum total probability of the product of the two PDFs at intermediate points.5. Confidence Determination computes the correlation confidence for each candidate pair using a two-stage machine learning algorithm that considers the kinematic feasibility, the scores from statistical object identification, and the initial confidence (where initial confidence does not consider other possible candidates) of all other possible candidates for the UCT. Once the confidence for a candidate pair exceeds an upper threshold, messages are generated for the correlated object.

[0097] As illustrated in FIG. 14, the workflow for post-maneuver UCT resolution can correlate UCTs to objects with fewer observations compared to traditional methods.

[0098] Finally, it will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the spirit or scope of the invention. For example, numerous materials, shapes, sizes and configurations can be substituted in place of those described herein. Thus, the present disclosure covers the modifications and variations provided they come within the scope of the appended claims and their equivalents..

Claims

WHAT IS CLAIMED:We claim:

1. A computer implemented method of determining characteristics of space objects, the method comprising: providing multiple space object data points over a period of time; processing the multiple space object data points using an analysis model to create a resident space object type score including the steps of: normalizing the multiple space object data points and removing unneeded data to create normalized data; processing the normalized data using at least one analysis model to create distinct feature data; and utilizing at least one set of distinct feature data set to produce a prediction of at least one characteristic of at least one space object and generating at least one space object type score; and generating a visual output corresponding to the at least one space object type score based on the distinct feature data.

2. The computer implemented method of determining the characteristics of space objects of claim 1 wherein the analysis model comprises at least one of a distinct set of statistical models.

3. The computer implemented method of determining the characteristics of space objects of claim 1 wherein the analysis model comprises a machine learning model.

4. The computer implemented method of determining the characteristics of space objects of claim 3 wherein the machine learning model is a convolutional neuralnetwork.

5. The computer implemented method of determining the characteristics of space objects of claim 1 wherein the space object data is ground-based.

6. The computer implemented method of determining the characteristics of space objects of claim 1 wherein the space object data is space-based.

7. The computer implemented method of determining the characteristics of space objects of claim 1 wherein the space object data further comprises space object meta-data.

8. The computer implemented method of determining the characteristics of space objects of claim 1 further comprising the step of providing an alert associated with a change in one or more photometric parameters.

9. The computer implemented method of determining the characteristics of space objects of claim 1 further comprising the step of receiving photometric data from a plurality of ground-based and / or space-based sensors.

10. The computer implemented method of determining the characteristics of space objects of claim 1 further comprising automated data processing.11 . The computer implemented method of determining the characteristics of space objects of claim 10 wherein the automated data processing further comprises generating an event probability output.

12. The computer implemented method of determining the characteristics of space objects of claim 10 wherein the automated data processing further comprises generating an object type probability output.

13. The computer implemented method of determining the characteristics of space objects of claim 1 wherein the analysis model processes the normalized data using at least one of: an Orbital Element Family ID model comprising an Orbital Element statistical model for building a distribution of object families based on orbital element information to output an In-Family object type score; a Photometric Family ID model comprising a Photometric Family statistical model for building a distribution of object families based on photometric information to output an In-Family object type score; an ML Target Classifier model comprising a machine learning model to output an In-Family object type score based on space object data; a Mission Type Family ID model comprising a Mission Type statistical model based on a normalized magnitude and a solar phase angle of a payload mission type to output an In-Family object type score; and an Area to Mass Ratio Family ID model for building a distribution of object families using a solar radiation pressure coefficient to output an In-Family object type score.

14. The computer implemented method of determining the characteristics of space objects of claim 13 wherein the Mission Type statistical model is a multidimensional statistical model.

15. The computer implemented method of determining the characteristics of space objects of claim 13 further comprising Class Violation Indicators.

16. The computer implemented method of determining the characteristics of space objects of claim 15, wherein the Class Violation I ndicator references an analysis model to determine whether a specific object falls within a distribution for set object types and compares the output to a public catalog designation, and wherein if the distribution for set object types and the public catalog designation are different, then outputting that the space object violates a class based on the analysis model.

17. The computer implemented method of determining the characteristics of space objects of claim 16 further comprising analyzing the Class Violation Indicators to generate a Target Nomination application.

18. The computer implemented method of determining the characteristics of space objects of claim 16, wherein a Target Nomination application combines the output of a plurality of Class Violation Indicators and compares a deviation of the space object from the plurality of Class Violation Indicators, then outputting a target nomination priority ranking based on a degree to which objects are violating its class.

19. A computer-implemented method for determining characteristics of space objects from data sources containing non-resolved electro-optical observation data over time, the method comprising the steps of: importing multiple data points over time from at least one of a plurality of sources of observation data; pre-processing the observation data to generate normalized data by: removing at least one of outlier, unneeded, and harmful data; normalizing the observation data to ensure consistency across sources; and calculating any necessary fields required by analysis models that are not present in the multiple data points; importing and pre-processing new observation data as it becomes availablefrom the data source; and processing the normalized data using at least one of a pre-determined statistical model and pre-trained machine learning model to generate a probability of at least one resident space object characteristic.

20. The computer-implemented method of claim 19 wherein the source of observation data is at least one of a database, a ground-based sensor, a plurality of ground-based sensors, an in-space sensor, and a plurality of in-space sensors.

21. The computer-implemented method of claim 19, wherein the preprocessing step further comprises identifying and excluding specific data points classified as outliers based on predefined criteria.

22. The computer-implemented method of claim 19, wherein the normalizing of data involves standardizing a format, units, and structure of data from various sources to create a unified dataset that isolates behavior of interest.

23. The computer-implemented method of claim 19, wherein calculating necessary fields includes deriving new metrics, aggregating data, or transforming existing data to meet the requirements of at least one analysis model.

24. The computer-implemented method of claim 23, wherein after transforming existing data it is fed into a convolutional neural network.

25. The computer-implemented method of claim 19 wherein the observation data comprises at least one of an apparent magnitude, a solar phase angle, an orbital semi-major axis, an orbital eccentricity, an orbital inclination, an orbital right ascension, an orbital declination, a solar radiation pressure coefficient, and an observation time.

26. The computer-implemented method of claim 19 wherein generating the at least one resident space object characteristic comprises an object classification report.

27. A system for determining characteristics of space objects, the system comprising at least one computing device comprising a central processing unit programmed with at least one application executable, wherein, when executed, causes the at least one computing device to: import multiple space object data points over a period of time; process the multiple space object data points using an analysis model to create a resident space object type score including the steps of: normalizing the multiple space object data points and removing unneeded data to create normalized data; processing the normalized data using at least one analysis model to create distinct feature data; utilizing at least one set of distinct feature data set to produce a prediction of at least one characteristic of the resident space object and generating the resident space object type score; and generating a visual output corresponding to the resident space object type based on the distinct feature data.

28. The system of claim 27 wherein the computing device is located in a ground control station on Earth.

29. The system of claim 27 wherein the computing device is located on a spacecraft in space.

30. The system of claim 27 wherein the analysis model comprises at leastone of a distinct set of statistical models.

31. The system of claim 27 wherein the analysis model comprises a machine learning model.

32. The system of claim 31 wherein the machine learning model is a convolutional neural network.

33. The system of claim 27 wherein the space object data further comprises space object meta-data.

34. The system of claim 27 further comprising the step of providing an alert associated with a change in one or more photometric parameters.

35. The system of claim 27 further comprising the step of receiving photometric data from a plurality of ground-based and / or space-based sensors.

36. The system of claim 27 further comprising automated data processing.

37. The system of claim 36 wherein the automated data processing further comprises generating an event probability output .

38. The system of claim 36 wherein the automated data processing further comprises generating an object type probability output.

39. The system of claim 27 wherein the analysis model processes the normalized data using at least one of:an Orbital Element Family ID model comprising an Orbital Element statistical model for building a distribution of object families based on orbital element information to output an In-Family object type score; a Photometric Family ID model comprising a Photometric Family statistical model for building a distribution of object families based on photometric information to output an In-Family object type score; an ML Target Classifier model comprising a machine learning model to output an In-Family object type score based on electro-optical observation data; a Mission Type Family ID model comprising a Mission Type statistical model based on a normalized magnitude and a solar phase angle of a payload mission type to output an In-Family object type score; and an Area to Mass Ratio Family ID model for building a distribution of object families using a solar radiation pressure coefficient to output an In-Family object type score.

40. The system of claim 39 wherein the Mission Type statistical model is a multi-dimensional statistical model.41 . The system of claim 39 further comprising Class Violation Indicators.

42. The system of claim 41 , wherein the Class Violation Indicator references an analysis model to determine whether a specific object falls within a distribution for set object types and compares the output to a public catalog designation, and wherein if the distribution for set object types and the public catalog designation are different, then outputting that the space object violates a class based on the analysis model.

43. The system of claim 42 further comprising analyzing the Class Violation Indicator to generate a Target Nomination application.

44. The system of claim 42, wherein a Target Nomination application combines the output of a plurality of Class Violation Indicators and compares a deviation of the space object from the plurality of Class Violation Indicators, then outputting a target nomination priority ranking based on a degree to which objects are violating its class.

Images