Systems, methods, and apparatus for performing on-platform analysis on collected images

Abstract

Solutions for performing on-platform analysis on collected images include: enriching the collected images with a packaged analysis component on an onboard orbital platform; selecting a set of collected images to be transmitted to a ground station based at least on the content of the collected images; and transmitting the selected set of collected images to the ground station. Other solutions include: packaging the analysis component for on-platform execution on an on-orbit platform; transmitting from the ground station to the platform at least one update selected from a list consisting of: an analysis algorithm update, a machine learning (ML) model, and ML training data; and executing the analysis component with the update. Other solutions include: performing intelligent compression on the collected images, wherein the intelligent compression process determines the data to be transmitted to the ground station based at least on the content of the collected images; and transmitting the data to the ground station.

Description

Background Art

[0001] In addition to defense and intelligence applications, satellites are an important data source for multiple verticals, including agriculture, forestry and land use planning, mining, oil and gas, retail forecasting, emergency management, and other commercial uses. The collected images are often enriched with analytics, such as by detecting objects in the image, and annotating the image for downstream users. The democratization of Earth Observation Data (EOD) from orbiting platforms has expanded the number of imaging platforms and the demand for imagery across multiple industries. This has brought challenges, such as higher bandwidth requirements for additional imagery that put pressure on communication channels.

[0002] Additionally, the potential for civilian assets to image sensitive military sites due to the lack of overflight restrictions on orbital platforms results in constraints that limit the resolution of imagery that can be transmitted down to Earth. Resolution limits are specified based on the ground sampling distance (GSD) at the Earth's surface, currently at some locations 0.25m for panchromatic imagery and 1.0m for multispectral GSD. Therefore, in some scenarios, some commercial orbital platforms may be required to downsample collected imagery to a lower resolution before transmitting it to a ground station. For example, an orbital platform may be launched with a sensor that is capable of providing higher resolution than currently allowed, resolution limits are expected to be relaxed in the future, or there may be a need to comply with multiple resolution requirements in different geopolitical jurisdictions. This may degrade potential analytical performance relative to that which could be achieved with raw imagery from onboard sensors. Summary of the invention

[0003] The disclosed examples are described in detail below with reference to the accompanying drawings listed below. The following summary is provided to illustrate some examples disclosed herein. However, this is not meant to limit all examples to any specific configuration or sequence of operations.

[0004] Some aspects disclosed herein relate to a solution for on-platform analysis, comprising: enriching collected images with a packaged analysis component on an onboard orbital platform; selecting a set of collected images to be transmitted to a ground station based at least on the content of the collected images; and transmitting the selected set of collected images to the ground station. Some aspects disclosed herein relate to a solution for on-platform analysis, comprising: packaging the analysis component for on-orbit platform execution on the on-orbit platform; transmitting from the ground station to the on-orbit platform at least one update selected from a list consisting of: analysis algorithm updates, machine learning (ML) models, and ML training data; and executing the analysis component with the update. Some aspects disclosed herein relate to a solution for on-platform analysis, comprising: enriching collected images with a packaged analysis component on an onboard orbital platform; performing an intelligent compression process on the collected images, wherein the intelligent compression process determines the data to be transmitted to the ground station based at least on the content of the collected images; and transmitting the data to the ground station. BRIEF DESCRIPTION OF THE DRAWINGS

[0005] The disclosed examples are described in detail below with reference to the following figures:

[0006] Figure 1 illustrates an arrangement that advantageously employs on-platform analysis;

[0007] Figure 2 A comparison of data flows between ground-based and on-platform analysis is illustrated;

[0008] Figure 3 A revisit time scenario for a rail platform is illustrated;

[0009] Figure 4 An exemplary process for configuring a track platform for on-platform analysis is illustrated;

[0010] Figure 5 Diagram showing the analytical capabilities on the updated platform;

[0011] Figure 6 An exemplary image is illustrated, along with a collection of detected objects (extracted entities);

[0012] Figure 7 is a flow chart illustrating exemplary operations involved in analysis on the platform;

[0013] Figure 8 is another flow chart illustrating exemplary operations involved in analysis on the platform; and

[0014] Fig. 9 is a block diagram of an example computing environment suitable for implementing some of the various examples disclosed herein.

[0015] Corresponding reference numerals indicate corresponding parts throughout the drawings. DETAILED DESCRIPTION

[0016] Various examples will be described in detail with reference to the accompanying drawings. Throughout the drawings, the same reference numerals will be used to refer to the same or similar parts where possible. Throughout this disclosure, references to specific examples and implementations are provided for illustrative purposes only, but are not intended to be limiting of all examples unless otherwise indicated.

[0017] In addition to defense and intelligence applications, satellites are an important data source for multiple verticals, including agriculture, forestry and land use planning, mining, oil and gas, retail forecasting, emergency management, and other commercial uses. The collected images are often enriched with analytics, such as by detecting objects in the image, and annotating the image for downstream users. The democratization of Earth Observation Data (EOD) from orbiting platforms has expanded the number of imaging platforms and the demand for imagery across multiple industries. This has brought challenges, such as higher bandwidth requirements for additional imagery that put pressure on communication channels.

[0018] Additionally, the potential for civilian assets to image sensitive military sites due to the lack of overflight restrictions on orbital platforms results in constraints that limit the resolution of imagery that can be transmitted down to Earth. Resolution limits are specified based on the ground sampling distance (GSD) at the Earth's surface, currently at some locations 0.25m for panchromatic imagery and 1.0m for multispectral GSD. Therefore, in some scenarios, some commercial orbital platforms may be required to downsample collected imagery to a lower resolution before transmitting it to a ground station. For example, an orbital platform may be launched with a sensor that is capable of providing higher resolution than currently allowed, resolution limits are expected to be relaxed in the future, or there may be a need to comply with multiple resolution requirements in different geopolitical jurisdictions. This may degrade potential analytical performance relative to that which could be achieved with raw imagery from onboard sensors.

[0019] Some satellites’ onboard sensors include multispectral and hyperspectral imagery (together referred to as “multispectral”). Because of the additional spectral information contained within the imagery, inferences can be made about the imaged scene. For example, the chemical composition and other properties of certain materials can be inferred, including soil moisture, pH, expected crop yield, and vegetation health. Furthermore, the additional information can improve the accuracy of object detection, recognition, and classification algorithms (together referred to as “object detection”) that are used to enrich the collected imagery. Analytics typically use artificial intelligence (AI) and / or machine learning (ML) algorithms (together referred to as “ML”) in object detection. However, the ML models used in ML engines require training. Generating training sets can be a labor-intensive process that requires selecting and annotating sufficiently large training datasets.

[0020] Aspects of the present disclosure operate in an unconventional manner to enrich collected imagery by performing on-platform analytics for an orbiting platform, such as a satellite. Aspects of the present disclosure operate in an unconventional manner to enhance on-platform analytics with synthetics, such as by training ML models with synthetics and / or by transmitting synthetic ML training data to an orbiting platform for onboard training. Aspects of the present disclosure operate in an unconventional manner to improve the transmission of collected imagery by performing intelligent compression and data selection to transmit imagery and / or data from an orbiting platform to a ground station.

[0021] Some aspects disclosed herein relate to a solution for on-platform analysis, comprising: enriching collected images with a packaged analysis component on an onboard orbiting platform; selecting a set of collected images to be transmitted to a ground station based at least on the content of the collected images; and transmitting the selected set of collected images to the ground station. Some aspects disclosed herein relate to a solution for on-platform analysis, comprising: packaging the analysis component for on-orbit execution by the on-orbit platform; transmitting from the ground station to the on-orbit platform at least one update selected from a list consisting of: an analysis algorithm update, a machine learning (ML) model, and ML training data; and executing the analysis component using the update.

[0022] Some aspects disclosed herein relate to solutions for on-platform analysis, including: enriching collected imagery with a packaged analysis component on an onboard orbital platform; performing an intelligent compression process on the collected imagery, wherein the intelligent compression process determines data to be transmitted to a ground station based at least on the content of the collected imagery; and transmitting the data to the ground station. When bandwidth is at a premium, some examples transmit data other than the imagery (such as a lightweight notification of an extracted entity) to the ground station, and transmit the imagery later when more bandwidth is available.

[0023] Figure 1 An arrangement that advantageously employs on-platform analysis is illustrated. An orbiting platform 100 (e.g., a satellite) images an imaging scene 160 on the surface of the Earth and transmits data to a ground station 152 across a communication channel 150. Some examples of communication channel 150 are similar to or identical to communication channel 926, which may be used to describe the communication channel 150. Fig. 9 100. The ground station 152 forwards data to a data repository 154 and receives uplink data, such as updates, missions, and commands for the platform 100. The data repository 154 is accessed by users 156, who retrieve imagery and data provided by the platform 100, and also provide updates, such as algorithm updates and updated ML models. In some examples, the data repository 154 provides a cloud-based service for managing components of the orbiting platform 100. The data repository 154 also interfaces with a synthesis service 158 to receive updates and training data, and provide feedback from any onboard ML training on the platform 100. Additional details are at Figure 5 Any of the ground station 152, the data repository 154, and the synthesis service 158 may be implemented on one or more instances of the computing device 900, the computing device 900 being related to Fig. 9 Described in more detail.

[0024] Platform 100 has at least one processor 102 and memory 110, which may be similar or identical to processor 914 and memory 912 of computing device 900, and thus, examples of platform 100 have at least some of the functionality described below for computing device 900. Platform 100 also has at least one sensor 104, and may have additional sensors 104. Some examples of sensors 104 include optical sensors, such as cameras, that collect visible light, whether red, green, and blue (RGB) color channels or other visible color channels. Some examples of sensors 104 include multi-spectral (including hyperspectral) sensors that collect light outside the visible spectrum. Platform 100 also has at least one transceiver 106 for communicating with ground stations 152 and other orbiting platforms 100a and 100b (see Figure 5 ).

[0025] Various data sets and logic (e.g., computer executable instructions) used by the processor 102 to perform on-platform analysis are stored in the memory 110. The sensor 104 outputs a raw image 112 based on the image capture capabilities of the sensor 104 (such as resolution and spectral band). The analysis component 114 ingests the raw image 112 to perform on-board analysis and detect objects and conditions (extracted entities) captured in the information available within the raw image 112. The analysis component 114 utilizes an ML component 120, such as an ML engine 122 that uses an ML model 124 to detect objects and conditions. In some examples, the ML model 124 includes multiple ML models (each ML model is a tensor) for detecting multiple different objects and conditions (e.g., extracting multiple different entities). The ML trainer 126 uses the ML training data 128 to perform on-board training to update or enhance the ML model 124 within the platform 100. Generally, since weight is a critical factor for rail platforms, and therefore lighter batteries are desired, executable programs such as analysis component 114 and ML component 120 will be designed for computational efficiency or selected from among various candidates based on computational efficiency.

[0026] The analysis component 114 outputs detected objects (extracted entities) 130, which are used to enrich the collection image. Figure 6When the original image 112 has a higher resolution than that allowed by transmission to the earth, the resolution limiting component 132 reduces the GSD of the original image 112 to produce a limited resolution image 134. When allowed, some examples perform the analysis component 114 on the higher resolution original image 112 instead of the limited resolution image 134, thereby potentially producing superior enriched results (e.g., superior data content within the detected object 130) than would be possible by performing the analysis on the limited resolution image 134.

[0027] Detected objects 130 and limited resolution images 134 are combined to produce enriched images 136. A subset of detected objects 130 and enriched images 136 are selected and compressed by intelligent compression component 140 for transmission to ground station 152. Communication component 142 manages the protocol, encryption, scheduling, and other aspects of the data flow from memory 110 to transceiver 106 for transmission of data to ground station 152. Mission data set 144 controls the mission to platform 100 and provides information for selecting a specific location schedule selected by the analysis algorithm, such as selecting an ML model that is more appropriate for imaging scene 160 (e.g., based on whether platform 100 is over land or over sea). In some examples, intelligent compression component 140 utilizes mission data set 144 to prioritize data for transmission to ground station 152.

[0028] Figure 2 A comparison of data streams 200a and 200b for ground-based analysis and on-platform analysis, respectively, is illustrated. Specifically, comparing data stream 200a to data stream 200b illustrates the advantages of on-platform analysis. In data stream 200a, imaged scene 160 is imaged by sensor 104 to produce raw image 112, which is stored in memory 110. When raw image 112 has a higher resolution than allowed for transmission to the earth, resolution limiting component 132 reduces the GSD of raw image 112 to produce limited resolution image 134. Limited resolution image 134 is stored in memory 110 for transmission over communication channel 150.

[0029] Relative to data stream 200b, data stream 200a conveys data of lower information density and places higher demands on bandwidth. Once arriving at data repository 154, ground-based analysis component 214 ingests limited resolution image 134 and outputs detected objects (extracted entities) 230. Detected objects 230 and limited resolution image 134 are combined to produce enriched image 236 for consumption by user 156.

[0030] In data flow 200b, imaged scene 160 is imaged by sensor 104 to produce raw image 112, which is stored in memory 110. Analysis component 114 ingests raw image 112 and outputs detected object 130. For scenes where analysis component 114 can use a higher resolution image (e.g., raw image 112 instead of limited resolution image 134), detected object 130 may have superior information content than detected object 230. When raw image 112 has a higher resolution than allowed for transmission to Earth, resolution limiting component 132 reduces the GSD of raw image 112 to produce limited resolution image 134. Detected object 130 and limited resolution image 134 are combined to produce enriched image 136. Enriched image 136 is stored in memory 110 for transmission over communication channel 150.

[0031] Relative to data stream 200a, data stream 200b delivers higher information density data and places lower bandwidth requirements when only a subset of the enriched images 136 are transmitted (via communication channel 150, via ground station 152) to data repository 154. For example, for data stream 200a, since the content of the generated limited resolution images 134 is unknown, the ground-based analysis component 214 receives and processes the entire set of limited resolution images 134. This requires that the entire set of limited resolution images 134 be transmitted across communication channel 150, even for scenes where limited resolution images 134 contain a large amount of repetitive information or contain few objects of interest. In contrast, for data stream 200b, if the detected object 130 indicates that the limited resolution image 134 or the enriched image 136 contains a large amount of repetitive information or few objects of interest, fewer images (e.g., only a subset of the enriched images 136) are transmitted across communication channel 150. Upon arrival at data repository 154, the enriched images 136 are available for consumption by user 156.

[0032] Figure 3 The diagram shows a revisit time scenario of an orbital platform. Satellite constellation 300 includes platform 100 and other orbital platforms 100a and 100b. Figure 3 As shown in FIG. 1 , the platform 100 is in a position to collect images of the imaging scene 160. The revisit cycle 302 (depicted by Figure 3 ) is the time required for the platform 100 to return in its orbital navigation path to a position where it can again collect images of the imaging scene 160. The inverse of the revisit period 302 is the revisit time of the platform 100.

[0033] However, some orbital imaging service providers abstract revisit times based on satellite constellations based on assumptions of interchangeability of imaging assets. Thus, for example, due to the orbital spacing of platform 100 and other orbital platforms 100a and 100b, the satellite constellation revisit period 304 is determined by how quickly one of the other orbital platforms 100a and 100b moves into position to begin collecting images of the imaging scene 160 after platform 100 can no longer view the imaging scene 160.

[0034] However, in the case where the analysis component 114 on the platform 100 uses the ML component 120, if the other orbiting platforms 100a and 100b do not have an exact copy of the ML component 120 (e.g., the ML model 124 has been uniquely trained), the capabilities of the analysis on the platform are not interchangeable. In such a scenario, using the revisit time of a specific satellite (e.g., platform 100) for mission planning may be more preferable than using the revisit time of a satellite constellation. Inter-satellite communications (which can share copies of the ML component (e.g., ML model 124) and with respect to Figure 5 ) may allow utilization of satellite constellation 300 for faster revisit times.

[0035] Figure 4 An exemplary process 400 for configuring the orbital platform 100 for on-platform analysis is illustrated. In some examples, the analysis component 114, including at least a portion of the ML component 120, is implemented on a field programmable gate array (FPGA) 402. In such examples, the FPGA 402 includes a portion of the memory 110. In some examples, an application specific integrated circuit (ASIC) is used in addition to or in place of the FPGA 402. Such a solution may be influenced by the stringent requirements for space-compliant hardware, including extensive safety requirements to ensure the safety of the platform 100 in harsh environments. In addition, the FPGA 402 is carefully configured to minimize power consumption, as weight and power budgets on orbital platforms are typically very strict.

[0036] One solution using FPGA 402 includes compressing the image collection and analysis engine into analysis component 114, programming FPGA 402 with it, and then installing FPGA 402 into platform 100. It should be understood that process 400 is one of many solutions for configuring orbital platform 100 for on-platform analysis, and other solutions (such as running analysis component 114 as instructions and data on processor 102) are also possible.

[0037] Figure 5Updating the on-board analytical capabilities is illustrated, updating various components such as analytical component 114 and ML component 120. The remoteness of platform 100 while on orbit creates challenges for updating analytical capabilities that are addressed as described herein. Once an operator or user of platform 100 determines that there is a need to update the onboard analytical capabilities of platform 100, the update is prepared. In some examples, this is an updated ML model 524a provided by user 156, which is sent via ground station 152 to update ML model 124 (see Figure 1 ). In some examples, this is an artificial synthetic trained ML model 524b, which is provided and trained by the synthesis service 158, which is sent via the ground station 152 to update the ML model 124. In examples employing onboard training, the synthesis service 158 provides synthetic ML training data 528 to enhance, update, or replace the ML training data 128. In some examples, the update includes other updates and fixes 502, which may, for example, include bug fixes in one or more of the analysis component 114, the ML engine 122, and the ML trainer 126. In some examples, the ML engine 122 and the ML trainer 126 are managed by a cloud-based service, such as through the data repository 154. Different platforms can be managed as different edge computing nodes, with different sets and focuses of artificial intelligence (AI) and ML algorithms.

[0038] Synthetic training provides the ability to cost-effectively improve analytical capabilities, enabling ML models to reason (e.g., detect objects, extract entities, etc.) using spectral bands available from multispectral sensors. This enables the use of information available within the imaged scene, even if the spectral information is outside the range of visible light. Synthesis service 158 also provides cost-effective training for images within visible bands (e.g., RGB). Some examples of synthesis service 158 can generate large amounts of ML training data faster than human experts.

[0039] Unfortunately, ground stations such as ground station 152 are geographically dispersed and may have full schedules. Therefore, to send any updates to platform 100, a communication session is scheduled between platform 100 and ground station 152 based on at least the desired navigation path of platform 100 and the location of ground station 152. Upon receiving the updates, platform 100 can execute analysis component 114 with the updates.

[0040] In some examples, platform 100 can also communicate with other orbital platforms 100a and 100b using communication channels 550a and 550b, respectively. This enables platform 100 to transmit (forward) at least a portion of the received updates to other orbital platforms 100a and 100b. After receiving the updates from platform 100, other orbital platforms 100a and 100b can utilize the updates to execute their own analysis components. This scenario presents satellite constellation 300 as a peer-to-peer network, which in some examples acts as a distributed computing arrangement.

[0041] Figure 6 An exemplary collected image 600 is illustrated, along with a collection of detected objects within an object mask 602. For example, the collected image 600 may represent an image within the original image 112 or the limited resolution image 134. The object mask 602 includes the size, location, and specific objects or conditions detected (extracted) in (from) the collected image 600. Information about the objects within the object mask is stored within the detected objects 130 and is used to enrich or annotate the collected image 600.

[0042] Figure 7 700 is a flowchart illustrating exemplary operations involved in intelligent configuration management of user equipment. In some examples, the operations described for flowchart 700 are performed by Fig. 9 The flowchart 700 begins at operation 702, which includes packaging the analysis component for on-orbit platform execution. In some examples, this includes implementing the analysis component software component in an FPGA. In some examples, this includes packaging the analysis component to operate within the environmental constraints of the orbital platform. Operation 704 includes placing the platform in orbit, and operation 706 includes collecting images by the platform to generate a collection of collected images. In some examples, this includes collecting the collected images using a multispectral sensor.

[0043] Operation 708 includes selecting at least one ML model to execute based at least on the platform location. For example, if the platform is over an ocean area, the usefulness of an analysis algorithm for detecting objects in a land-based setting is limited. Therefore, alternatively, an analysis algorithm suitable for multispectral images of ocean scenes, such as determining salinity, temperature, and aquatic life concentration, will be selected. In some examples, the orbital position of the platform is used as a factor in selecting the ML model to be executed. The orbital position can affect the amount of power available to the solar panels, such as when the platform is in the shadow of the earth. This means that battery power is required for processing. Some ML models that require relatively intensive processing cause the processor to place higher power demands on the platform's power reserves (such as batteries). Therefore, in some examples, when the platform is in the shadow and running on batteries, ML models that produce higher power consumption will not be selected for execution, while in other cases where the platform is receiving full solar energy or the battery reserves are sufficient, the ML model will be selected for execution.

[0044] Typically, because weight is a very critical factor for orbital platforms, thereby driving the need to use lighter weight batteries, the ML models selected for use on the platform will be designed for computational efficiency. As an example of reducing processor requirements, RGB-based image classification or object detection / recognition algorithms can be avoided when the platform is on the shadowed (nighttime) side of the earth and collecting infrared (IR) images. Operation 710 includes performing analysis on the collected images using a packaged analysis component, for example, executing the ML model selected in operation 708. Operation 712 includes reducing the resolution of the collected images to produce limited resolution images. This is performed in some examples to comply with legal requirements for specifying a GSD.

[0045] Operation 714 includes enriching the collected images on the orbiting platform using the package analysis component. In some examples, this involves annotating the limited resolution images, although in some examples, the original resolution images are annotated. Operation 716 includes selecting a set of collected images to be transmitted to the ground station based at least on the content of the collected images; and operation 718 includes performing an intelligent compression process on the collected images. In some examples, the intelligent compression process determines the data to be transmitted to the ground station based at least on the content of the collected images. That is, intelligence is involved in ensuring spectral efficiency (e.g., determining how much collection and enrichment data to send). Different ground stations may have different data ingestion capabilities, so the intelligent compression process selects the amount of data to be sent based at least on the capabilities of the ground station. Some examples only send an identification of the image content (e.g., detected objects or conditions) and send collected images containing certain detected objects or conditions. Some examples identify images containing repeated enriched data (e.g., detected objects or conditions) and send only a single example to avoid sending unnecessary duplicates.

[0046] Based on the data transmission profile determined in decision operation 720, operation 722 includes transmitting the selected collection image set to the ground station. In some examples, the selected collection image set transmitted to the ground station includes at least a portion of the enriched image enriched in operation 714. Alternatively, operation 724 includes transmitting data other than the image to the ground station. In some examples, this includes transmitting inference data (enriched data) for a limited set (less than all) of frequency bands from the multispectral collection image set. That is, when bandwidth is very precious, some examples send lightweight notifications of extraction entities and transmit images later when more bandwidth is available.

[0047] Figure 8 8000 is a flowchart illustrating exemplary operations involved in intelligent configuration management of user equipment. In some examples, the operations described for flowchart 800 are performed by Fig. 9 Executed by computing device 900. Flowchart 800 begins with operations 702-706, as described above for Figure 7 Described. Operation 802 includes determining the need for an update to the packaged analysis component. In some examples, this includes an analysis algorithm update, such as a bug fix or other maintenance. Operation 804 includes determining the need for a new ML model. In some examples, the new ML model is added to the existing set of ML models on the platform. In some examples, the new ML model is an update that replaces one of the existing ML models on the platform. In some examples, the new ML model is trained by a synthesis service before transmission. Operation 806 includes determining the need for ML training data. In this way, the machine learning model on the platform can be updated using the training on the platform. In some examples, the synthesis service provides ML training data with synthetic training data. Both the new ML model and the ML training data are updates to the analysis component because the analysis component includes an ML component. Therefore, operations 802-806 together include determining the need for an update to the packaged analysis component, wherein the update to the packaged analysis component includes: at least one update selected from a list consisting of: an analysis algorithm update, a new ML model, and ML training data.

[0048] Operation 808 includes determining the platform availability for communication. For example, this may include determining the ground station communication capabilities (e.g., the next ground station to visit) based at least on the expected navigation path of the platform. Operation 810 includes scheduling communications with the platform based at least on the expected navigation path of the platform. Operation 812 includes transmitting at least one update from the ground station to the on-orbit platform, the at least one update selected from a list consisting of: an analysis algorithm update, an ML model update, and an ML training data update. In some examples, the update (ML model update) replaces one of the existing ML models on the platform. In some examples, the update (ML training data update) includes synthesized training data. Operation 814 includes executing the analysis component using the update, such as executing operation 710 of flowchart 700. Operation 816 includes transmitting (forwarding) at least a portion of the update from the platform to a second orbiting platform. Operation 818 includes executing the update using the analysis component on the second orbiting platform.

[0049] Additional Examples

[0050] Some aspects and examples disclosed herein relate to a system for performing on-platform analysis on collected images, the system comprising: a processor; and a computer-readable medium storing instructions that, when executed by the processor, are operable to: enrich the collected images using a packaged analysis component on an onboard orbital platform; select a set of collected images to be transmitted to a ground station based at least on the content of the collected images; and transmit the selected set of collected images to the ground station.

[0051] Additional aspects and examples disclosed herein relate to a system for performing on-platform analysis of collected imagery, the system comprising: a processor; and a computer-readable medium storing instructions that, when executed by the processor, are operable to: package an analysis component for on-platform execution on an on-orbit platform; transmit at least one update from a ground station to the platform, the at least one update selected from a list consisting of: an analysis algorithm update, an ML model update, and an ML training data update; and execute the analysis component using the update.

[0052] Additional aspects and examples disclosed herein relate to a method for performing on-platform analysis on collected images, the method comprising: enriching the collected images using a packaged analysis component on an onboard orbital platform; selecting a set of collected images to be transmitted to a ground station based at least on the content of the collected images; and transmitting the selected set of collected images to the ground station.

[0053] Additional aspects and examples disclosed herein relate to a method of performing on-platform analysis on collected imagery, comprising: packaging an analysis component for on-platform execution on an on-orbit platform; transmitting at least one update from a ground station to the platform, the at least one update selected from a list consisting of: an analysis algorithm update, an ML model update, and an ML training data update; and executing the analysis component using the update.

[0054] Additional aspects and examples disclosed herein relate to one or more computer storage devices having stored thereon computer executable instructions for performing on-platform analysis of collected images, the computer executable instructions, when executed by a computer, causing the computer to perform operations including: enriching the collected images on an onboard orbital platform using a packaged analysis component; selecting a set of collected images to be transmitted to a ground station based at least on the content of the collected images; and transmitting the selected set of collected images to the ground station.

[0055] Additional aspects and examples disclosed herein relate to one or more computer storage devices having stored thereon computer executable instructions for performing on-platform analysis of collected imagery, the computer executable instructions, when executed by a computer, causing the computer to perform operations comprising: packaging an analysis component for on-platform execution on an on-orbit platform; transmitting at least one update from a ground station to the platform, the at least one update selected from a list consisting of: analysis algorithm updates, ML model updates, and ML training data updates; and executing the analysis component using the update.

[0056] Alternatively or in addition to other examples described herein, examples include any combination of the following:

[0057] - performing an intelligent compression process on the collected image, wherein the intelligent compression process determines data to be transmitted to the second ground station based at least on the content of the collected image;

[0058] - transmit data to a second ground station;

[0059] - the data includes extrapolated data from a multispectral collection image set for a limited set of frequency bands less than all frequency bands;

[0060] -Collecting image sets using multispectral sensors;

[0061] - based at least on the platform location, selecting at least one ML model to execute;

[0062] - Enriching the collected images includes: performing analysis on the collected images using the selected ML model;

[0063] - reducing the resolution of the collected images to produce limited resolution images;

[0064] - transmitting the selected set of collected images to a ground station comprises: transmitting a limited resolution image set;

[0065] -Enriching the image collection including annotation of limited resolution images;

[0066] - Implement the analysis component software component in FPGA;

[0067] -Encapsulate the analysis component orbit platform environmental constraints;

[0068] -Put the platform on the track;

[0069] - the selected set of collected images transmitted to the ground station includes at least a portion of the enriched image;

[0070] - Identify the need for updates to the package analysis components;

[0071] - Determine the availability of platforms for communication;

[0072] - scheduling communications with the platform based at least on the desired navigation path of the platform;

[0073] - Update and replace existing ML models on the platform;

[0074] - Updates include synthetic training data;

[0075] - transmitting at least a portion of the update from the platform to a second orbital platform;

[0076] - Perform updates using the analytics component on the second track platform;

[0077] - collect images through the Platform; and

[0078] - Perform analysis on the collected images using package analysis components.

[0079] Although aspects of the disclosure have been described in terms of various examples and their associated operations, those skilled in the art will appreciate that combinations of operations from any number of the different examples are also within the scope of aspects of the disclosure.

[0080] Example operating environment

[0081] Fig. 9900 is a block diagram of an example computing device 900 for implementing aspects disclosed herein, and is generally designated as computing device 900. Computing device 900 is only an example of a suitable computing environment, and is not intended to imply any limitation on the scope of use or functionality of the examples disclosed herein. Computing device 900 should also not be interpreted as having any dependency or requirement on any one component / module or combination of components / modules described. The examples disclosed herein can be described in the general context of computer code or machine-usable instructions, including computer executable instructions executed by a computer or other machine (such as a personal data assistant or other handheld device), such as program components. Typically, program components (including routines, programs, objects, components, data structures, etc.) refer to codes that perform specific tasks or implement specific abstract data types. The disclosed examples can be practiced in various system configurations, including personal computers, laptop computers, smart phones, mobile tablets, handheld devices, consumer electronics, professional computing devices, etc. When tasks are performed by remote processing devices linked through a communication network, the disclosed examples can also be practiced in a distributed computing environment.

[0082] The computing device 900 includes a bus 910 that directly or indirectly couples the following devices: computer storage memory 912, one or more processors 914, one or more presentation components 916, I / O ports 918, I / O components 920, power supply 922, and network components 924. Although the computing device 900 is depicted as a seemingly single device, multiple computing devices 900 can work together and share the depicted device resources. For example, the memory 912 can be distributed across multiple devices, and the processor 914 can be housed with different devices.

[0083] Bus 910 represents what may be one or more buses (such as an address bus, a data bus, or a combination thereof). Although shown with lines for clarity, Fig. 9 , depicting various components may be accomplished using alternative representations. For example, in some examples, presentation components such as display devices are I / O components, and some examples of processors have their own memory. No distinction is made between categories such as "workstation," "server," "laptop," "handheld device," etc., as all are expected to be in the Fig. 9and within the scope of references to "computing device" herein. Memory 912 may take the form of computer storage media referenced below and is operable to provide storage of computer-readable instructions, data structures, program modules, and other data for computing device 900. In some examples, memory 912 stores one or more of: an operating system, a general-purpose application platform, or other program modules and program data. Thus, memory 912 is capable of storing and accessing data 912a and instructions 912b, which are executable by processor 914 and configured to implement various operations disclosed herein.

[0084] In some examples, memory 912 includes computer storage media in the form of volatile and / or non-volatile memory, removable or non-removable memory, data disks in a virtual environment, or a combination thereof. Memory 912 may include any number of memories associated with or accessible by computing device 900. Memory 912 may be internal to computing device 900 (e.g., Fig. 9 ), external to computing device 900 (not shown), or both (not shown). Examples of memory 912 include, but are not limited to, random access memory (RAM); read-only memory (ROM); electronically erasable programmable read-only memory (EEPROM); flash memory or other memory technology; CD-ROM, digital versatile disk (DVD) or other optical or holographic media; cassettes, magnetic tape, disk storage or other magnetic storage devices; memory wired to an analog computing device; or any other medium for encoding the desired information and accessed by computing device 900. Additionally or alternatively, memory 912 may be distributed across multiple computing devices 900, for example, in a virtualized environment where instruction processing is performed on multiple devices 900. For purposes of this disclosure, "computer storage media," "computer storage memory," "memory," and "memory device" are synonymous terms for computer storage memory 912, and none of these terms include carrier waves or propagated signals.

[0085] (Multiple) processors 914 may include any number of processing units that read data from various entities (such as memory 912 or I / O components 920). Specifically, (multiple) processors 914 are programmed to execute computer executable instructions to implement aspects of the present disclosure. Instructions can be executed by a processor, by multiple processors within the computing device 900, or by a processor outside the client computing device 900. In some examples, the processor 914 is programmed to execute instructions, such as those illustrated in the flowcharts discussed below and depicted in the accompanying drawings. In addition, in some examples, the processor 914 represents an implementation of simulation technology for performing the operations described herein. For example, the operations can be performed by a simulated client computing device 900 and / or a digital client computing device 900. The presentation component 916 presents data indications to a user or other device. Exemplary presentation components include display devices, speakers, printing components, vibration components, etc. Those skilled in the art will understand and appreciate that computer data can be presented in a variety of ways, such as visually presented in a graphical user interface (GUI), audibly presented through a speaker, wirelessly presented between computing devices 900, presented across a wired connection, or presented in other ways. I / O ports 918 allow computing device 900 to be logically coupled to other devices including I / O components 920, some of which may be built-in. Example I / O components 920 include, for example but not limited to, microphones, joysticks, game pads, satellite dishes, scanners, printers, wireless devices, etc.

[0086] The computing device 900 can operate in a networked environment using logical connections to one or more remote computers via a network component 924. In some examples, the network component 924 includes a network interface card and / or computer executable instructions (e.g., a driver) for operating a network interface card. Communications between the computing device 900 and other devices can be performed over any wired or wireless connection using any protocol or mechanism. In some examples, the network component 924 can be operable to communicate wirelessly using short-range communication technologies (e.g., near field communication (NFC), Bluetooth TM The network component 924 communicates with the cloud resources 928 across the network 930 via wireless communication channels 926 and / or wired communication channels 926a. Various different examples of communication channels 926 and 926a include wireless connections, wired connections, and / or dedicated links, and in some examples, at least a portion is routed through the Internet.

[0087] Although described in conjunction with the example computing device 900, the examples of the present disclosure can be implemented using many other general or special computing system environments, configurations, or devices. Examples of well-known computing systems, environments, and / or configurations that may be suitable for use with aspects of the present disclosure include, but are not limited to, smartphones, mobile tablets, mobile computing devices, personal computers, server computers, handheld or laptop devices, multiprocessor systems, game consoles, microprocessor-based systems, set-top boxes, programmable consumer electronics, mobile phones, mobile computing and / or communication devices in wearable or accessory form factors (e.g., watches, glasses, headphones, or earpieces), network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, virtual reality (VR) devices, augmented reality (AR) devices, mixed display (MR) devices, holographic devices, etc. Such systems or devices can accept input from a user (including from an input device such as a keyboard or pointing device) in any manner via gesture input, proximity input (such as by hovering), and / or via voice input.

[0088] Examples of the present disclosure may be described in the general context of computer executable instructions such as program modules executed by one or more computers or other devices in software, firmware, hardware or a combination thereof. Computer executable instructions may be organized into one or more computer executable components or modules. Typically, program modules include, but are not limited to, routines, programs, objects, components, and data structures that perform specific tasks or implement specific abstract data types. Aspects of the present disclosure may be implemented using any number and organization of such components or modules. For example, aspects of the present disclosure are not limited to specific computer executable instructions or specific components or modules shown in the figures and described herein. Other examples of the present disclosure may include different computer executable instructions or components with more or less functionality than that shown and described herein. In an example involving a general-purpose computer, when configured to execute instructions described herein, aspects of the present disclosure convert the general-purpose computer into a special-purpose computing device.

[0089] As an example and not limitation, computer-readable media include computer storage media and communication media. Computer storage media include volatile and non-volatile memory, removable and non-removable memory implemented in any method or technology for storing information such as computer-readable instructions, data structures, program modules, etc. Computer storage media are tangible media and are mutually exclusive with communication media. Computer storage media are implemented in hardware and exclude carrier waves and propagation signals. For the purposes of this disclosure, computer storage media themselves are not signals. Exemplary computer storage media include hard disks, flash drives, solid-state memories, phase change random access memories (PRAM), static random access memories (SRAM), dynamic random access memories (DRAM), other types of random access memories (RAM), read-only memories (ROM), electrically erasable programmable read-only memories (EEPROM), flash memory or other memory technologies, compact disk read-only memories (CD-ROM), digital versatile disks (DVD) or other optical storage devices, cassettes, tapes, disk storage devices or other magnetic storage devices, or any other non-transmission media that can be used to store information for access by a computing device. In contrast, communication media typically embodies computer readable instructions, data structures, program modules, etc. in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media.

[0090] The execution (execution) or the order of execution (performance) of the operation in the example of the present disclosure shown and described herein is not necessary, and can be executed in different sequential ways in various examples.For example, it is expected that before, simultaneously or afterwards another operation, executing (executing) or performing (performing) a specific operation is within the scope of the various aspects of the present disclosure.When introducing the elements of various aspects of the present disclosure or its examples, the articles "a", "an", "the" and "said" are intended to represent one or more elements in the element.The terms "comprising", "including" and "having" are intended to be inclusive, and mean that there may be additional elements in addition to the listed elements.The term "exemplary" is intended to mean "an example of...". The phrase "one or more of the following: A, B and C" means "at least one of A and / or at least one of B and / or at least one of C".

[0091] Having described various aspects of the disclosure in detail, it will be apparent that modifications and variations are possible without departing from the scope of various aspects of the disclosure as defined in the appended claims. As various changes may be made to the above-described constructions, products and methods without departing from the scope of various aspects of the disclosure, it is intended that all matter contained in the above description and shown in the accompanying drawings shall be interpreted as illustrative rather than in a limiting sense.

Claims

1. A system for performing on-platform analysis on collected images, the system comprising: processor; as well as A computer readable medium storing instructions which, when executed by the processor, are operable to: packaging the analysis components for on-orbit platform execution; determining a need for an update of the analysis component of the package; Determine the availability of platforms for communication; scheduling communications with the platform based at least on a desired navigation path of the platform; At least one update is transmitted from a ground station to the on-orbit platform, the at least one update being selected from a list consisting of: Analysis algorithm updates, machine learning ML model updates, and ML training data updates, wherein at least one of the updates replaces an existing ML model onboard the platform, and the platform executes an analysis component utilizing the at least one update.

2. The system of claim 1, wherein the instructions are further operable to: Training data is received to update the analysis component of the package.

3. The system of claim 1, wherein the at least one update comprises synthetic training data.

4. The system of claim 1, wherein the instructions are further operable to: At least a portion of the at least one update is communicated from the platform to a second orbital platform. 5 . The system of claim 4 , wherein the second track platform utilizes the at least one update to execute a second analysis component of the second track platform.

6. The system of claim 1, wherein the instructions are further operable to: collecting images by the platform; and Using the analysis component of the package, analysis is performed on the collected images.

7. A method for performing on-platform analysis on a collected image, the method comprising: packaging the analysis components for on-orbit platform execution; determining a need for an update of the analysis component of the package; Determine the availability of platforms for communication; scheduling communications with the platform based at least on a desired navigation path of the platform; At least one update is transmitted from a ground station to the on-orbit platform, the at least one update being selected from a list consisting of: Analysis algorithm updates, machine learning ML model updates, and ML training data updates, wherein at least one of the updates replaces an existing ML model onboard the platform, and the platform executes an analysis component utilizing the at least one update. The method of claim 7 , wherein the at least one update comprises synthetic training data.

9. The method according to claim 7, further comprising: At least a portion of the at least one update is communicated from the platform to a second orbital platform.

10. The method according to claim 9, wherein: The second track platform utilizes the at least one update to execute a second analysis component of the second track platform.

11. The method according to claim 7, further comprising: collecting images by the platform; as well as Analysis is performed on the collected images using the analysis component of the package.

12. One or more computer storage devices having stored thereon computer executable instructions for performing on-platform analysis of collected images, the instructions, when executed by a computer, causing the computer to perform operations comprising: packaging the analysis components for on-orbit platform execution; determining a need for an update of the analysis component of the package; Determine the availability of platforms for communication; scheduling communications with the platform based at least on a desired navigation path of the platform; At least one update is transmitted from a ground station to the platform, the at least one update being selected from a list consisting of: Analysis algorithm updates, machine learning ML model updates, and ML training data updates, wherein at least one of the updates replaces an existing ML model onboard the platform, and the platform executes an analysis component utilizing the at least one update.

13. The one or more computer storage devices of claim 12, wherein the operations for determining the need for the update further comprise: Data is received from the on-orbit platform.

14. The one or more computer storage devices of claim 12, wherein the at least one update comprises synthetic training data.

15. The one or more computer storage devices of claim 12, wherein the operations further comprise: transferring at least a portion of the at least one update from the platform to a second orbital platform, The second track platform utilizes the at least one update to execute a second analysis component of the second track platform.

16. The one or more computer storage devices of claim 12, wherein the operations further comprise: collecting images by the platform; as well as Using the analysis component of the package, analysis is performed on the collected images.

Images