System operable to electronically monitor mission

Abstract

To provide a system for monitoring a mission.SOLUTION: The system includes a digital ecosystem (DE) associated with the mission and positioned at a remote station. In response to sensed data associated with deployed products positioned locally with the mission, the DE generates performance data associated with the deployed products. A processor includes local processor resources located at the mission, and remote processor resources at the remote station. Operations of the processor include: executing a first cognitive algorithm which, in response to a portion of the performance data, generates an initial cognitive output associated with an initial cognitive output action; prior to initiating the initial cognitive output action associated with the initial cognitive output, displaying to a user the initial cognitive output; and, in response to receiving user feedback on the initial cognitive output, executing post-user-feedback operations based on the user feedback.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] (CROSS REFERENCE TO RELATED APPLICATIONS) This application claims the benefit of U.S. Provisional Application No. 63 / 518,766, filed August 10, 2023, and U.S. Provisional Application No. 63 / 607,737, filed December 8, 2023, the disclosures of which are incorporated by reference in their entireties herein.

[0002] The present disclosure relates generally to electronic analysis of computer-controlled distributed products and systems, and more particularly to a novel cognitive learning-based mission analysis and control system operable to automatically monitor and analyze data to detect trends and perform problem solving. In an embodiment of the present disclosure, the novel cognitive learning-based mission analysis and control system is further operable to incorporate and capture human expertise that can be used to extend the self-learning capabilities. [Background technology]

[0003] As used herein, the term "mission" is used to refer to any activity having a goal that generates telemetry (i.e., telemetry data) in the process of achieving some intended goal. In general, telemetry data refers to the collection and transmission of measurements from a source using, for example, sensors and protocols. Missions often involve the use of some form of mobile or fixed structure, including, but not limited to, semi-trucks, construction vehicles, office buildings, airplanes, trains, hospital beds, submarines, and the like. In some situations, a mission may involve human exploration of the surroundings, including travel to unknown, dangerous, or difficult to access areas to discover and learn. Such missions may occur on land with any type of terrain (e.g., mountains, caves, etc.), underground at various depths, in water at various depths, in the air within the Earth's atmosphere, and in outer space beyond the Earth's atmosphere.

[0004] A non-limiting example of a mission is space exploration. Space development is categorized into sectors such as civil, national security (i.e. defense and intelligence), and commercial. Each sector operates with its own goals and assets, although they all rely on a common space industrial base, workforce, and infrastructure. The civil space sector generally covers government space activities that are not defense-related, including launching satellites, managing satellites, conducting research, and exploring the solar system. In the United States, nearly all civil space missions are managed or performed by the National Aeronautics and Space Administration (NASA) and the National Oceanic and Atmospheric Administration (NOAA).

[0005] The national security space sector encompasses both the defense and intelligence sectors. The Department of Defense oversees space missions in support of military operations, and several agencies of the U.S. intelligence community are involved in employing space assets for intelligence purposes in support of military and law enforcement operations.

[0006] The commercial space sector generally includes goods, services, and activities provided by private companies with the legal capacity to provide products to non-governmental customers. Examples of commercial uses of space include satellite navigation, satellite television, and commercial satellite imagery. Providers of such services typically outsource the manufacture of satellites and their launch to private or public companies, which form an integral part of the space economy. Commercial space endeavors are expanding rapidly.

[0007] The computing systems, mechanical systems, and know-how required to safely carry out and monitor mission control activities in general, and space activities in particular, are extensive, complex, and often distributed, making it difficult to provide relatively large amounts of power, computing resources, and personnel for such mission control activities. Summary of the Invention [Problem to be solved by the invention]

[0008] SUMMARY OF THE DISCLOSURE The object of the present invention is therefore to improve a system of the kind mentioned at the outset, so that the above-mentioned disadvantages are reduced. [Means for solving the problem]

[0009] A system for monitoring a mission is disclosed. The system includes a processor system in communication with a digital ecosystem (DE) located at a remote station. The DE generates performance data for the deployed product in response to sensory data for the deployed product located locally with the mission. The processor system includes a local processor system resource located at the mission as well as a remote processor resource at a remote station located remotely from the mission. The processor system performs processor system operations including: executing a first cognitive algorithm in response to a portion of the performance data to generate an initial cognitive output associated with an initial cognitive output operation; displaying the initial cognitive output to a user prior to initiating the initial cognitive output operation; and, in response to receiving user feedback regarding the initial cognitive output, performing a post-user feedback operation based at least in part on the user feedback.

[0010] In addition to any one or more of the features described herein, the first cognitive algorithm includes one or more of an anomaly detection cognitive algorithm, a prognosis cognitive algorithm, and a performance evaluation cognitive algorithm.

[0011] In addition to any one or more of the features described herein, the user feedback includes the user's agreement with the initial cognitive output.

[0012] In addition to any one or more of the features described herein, the operation after user feedback includes initiating an initial cognitive output action.

[0013] In addition to any one or more of the features described herein, the user feedback includes altering the initial cognitive output.

[0014] In addition to any one or more of the features described herein, the user feedback includes modifying the initial cognitive output behavior.

[0015] In addition to any one or more of the features described herein, the user feedback includes modifying the initial cognitive output and modifying the initial cognitive output behavior.

[0016] In addition to any one or more of the features described herein, the operations after the user feedback include using the user feedback to train a user model, which includes an electronic model of the user.

[0017] In addition to any one or more of the features described herein, the user model is trained to perform a task including generating user-model-generated feedback including a prediction of feedback that would be generated by the user in response to an instance of the initial cognitive output.

[0018] In addition to any one or more of the features described herein, the user model includes a digital twin of the user.

[0019] In addition to any one or more of the features described herein, the first cognitive algorithm includes one or more of an anomaly detection algorithm, a prediction algorithm, and a performance evaluation algorithm.

[0020] In addition to any one or more of the features described herein, the operations of the processor system further include executing a second cognitive algorithm to evaluate sufficiency of the user feedback.

[0021] In addition to any one or more of the features described herein, sufficiency of the user feedback includes determining whether the user feedback includes a rationale for the user feedback and whether the rationale meets one or more predetermined criteria.

[0022] In addition to any one or more of the features described herein, the second cognitive algorithm issues a query to the user in response to a determination by the second cognitive algorithm that the user feedback is insufficient.

[0023] In addition to any one or more of the features described herein, the query requests a rationale for user feedback from the user.

[0024] In addition to any one or more of the features described herein, the query requests a rationale supplement of user feedback from the user.

[0025] In addition to any one or more of the features described herein, the second cognitive algorithm includes a question and answer algorithm having natural language processing capabilities, and the user feedback includes a natural language rationale for the user feedback.

[0026] In addition to any one or more of the features described herein, the operation of the processor system further includes generating a knowledge corpus of the user by storing results of the operations after the user feedback in a data repository.

[0027] Also disclosed is a system including a processor system electronically coupled to a memory, the processor system performing processor system operations including accessing actual data associated with a plurality of components. The actual data is provided to a physics-based perception algorithm operable to simulate performance of a first component. The physics-based perception algorithm is executed to generate enhanced simulated performance data for the first component based at least in part on the actual data associated with the plurality of components.

[0028] In addition to any one or more of the features described herein, the first component is distinct from the plurality of components.

[0029] In addition to any one or more of the features described herein, the physics-based perception algorithm for generating the enhanced simulated performance data for the first component based at least in part on the actual data associated with the plurality of components includes retraining the physics-based perception algorithm based at least in part on the actual data associated with the plurality of components.

[0030] In addition to any one or more of the features described herein, the physics-based cognitive algorithms include, but are not limited to, generative neural networks.

[0031] In addition to any one or more of the features described herein, operations of the processor system further include receiving a first operating condition for the first component, and generating a version of the enhanced simulation performance data for the first component using the first operating condition and the enhanced simulation performance data for the first component that simulates performance data for the first component that would exist under the first operating condition.

[0032] In addition to any one or more of the features described herein, the operations of the processor system further include generating a version of the enhanced simulation performance data of the first component that simulates performance data of the first component as it would exist under the first operating conditions using the actual data associated with the plurality of components, the first operating conditions, and the enhanced simulation performance data of the first component.

[0033] In addition to any one or more of the features described herein, the first operational condition of the first component includes a fault condition of the first component.

[0034] In addition to any one or more of the features described herein, the first operational condition of the first component is selected from the group consisting of a fault condition of the first component, a predicted fault condition of the first component, a degraded performance condition of the first component, application of an environmental stressor to the first component, and an emergency operational mode of the first component.

[0035] In addition to any one or more of the features described herein, the operations of the processor system further include using the cognitive algorithm to generate a version of the enhanced simulation performance data of the first component using the first operating conditions and the enhanced simulation performance data of the first component that simulates the performance data of the first component as it would exist under the first operating conditions.

[0036] In addition to any one or more of the features described herein, operation of the processor system further includes providing training data to the cognitive algorithm, the training data including a version of the generated enhanced simulation performance data of the first component that simulates performance data of the first component that would exist under said first operating conditions.

[0037] In addition to any one or more of the features described herein, the training data further includes actual data relating to the plurality of components.

[0038] In addition to any one or more of the features described herein, the training data further includes user feedback regarding the cognitive output generated by the cognitive algorithm.

[0039] In addition to any one or more of the features described herein, the perception algorithm is selected from the group consisting of a predictive perception algorithm, an anomaly detection perception algorithm, and a performance analysis perception algorithm.

[0040] Further disclosed is a system including a processor system electronically connected to a memory. The processor system executes processor system operations including executing a first cognitive algorithm to generate an initial cognitive output associated with an initial cognitive output operation. Prior to commencing the initial cognitive output operation associated with the initial cognitive output, the initial cognitive output is displayed to a user. In response to receiving user feedback regarding the initial cognitive output, a post-user feedback operation is executed based at least in part on the user feedback. Actual data associated with a plurality of components is accessed. The actual data is provided to a physics-based cognitive algorithm operable to simulate performance of the first component. The physics-based cognitive algorithm is executed to generate enhanced simulated performance data for the first component based at least in part on the actual data associated with the plurality of components. A first operating condition of the first component is received. Using the first operating condition and the enhanced simulated performance data of the first component, a version of the enhanced simulated performance data of the first component is generated that simulates performance data of the first component that would exist under the first operating condition. The processor system operations further include providing training data to the cognitive algorithm. The training data further includes a version generating enhanced simulated performance data of the first component simulating performance data of the first component that would exist under the first operating conditions, actual data associated with the plurality of components, and user feedback regarding the initial cognitive output.

[0041] Further disclosed is a system operable to electronically monitor a Near Earth Space Exploration (NESE) mission. The system includes a processor system associated with the NESE mission and in electronic communication with a digital ecosystem (DE) located locally with the NESE mission. The DE is operable to generate performance data associated with the deployed product in response to sensed data associated with the deployed product located locally with the NESE mission. The processor system includes a local processor system resource located locally with the NESE mission. The processor system further includes a remote processor resource at a remote station located on Earth. The processor system is operable to perform processor system operations including: executing a first cognitive algorithm operable to generate an initial cognitive output associated with an initial cognitive output operation in response to a portion of the performance data; displaying the initial cognitive output to a user located locally with the NESE mission or located at a remote station prior to initiating an initial cognitive output operation associated with the initial cognitive output; and performing a post-user feedback operation based at least in part on the user feedback in response to receiving user feedback regarding the initial cognitive output. In some embodiments of the present disclosure, a processor system, digital ecosystem, etc., is located at a remote station and is operable to receive telemetry information (e.g., sensed data) from deployed products and other equipment located locally by the NESE mission.

[0042] In addition to any one or more of the features described herein, the deployed product includes a first deployed product, the sensed data includes first sensed data associated with the first deployed product, and the operation of the processor system further includes executing a physics-based perception algorithm operative to simulate performance of the first deployed product in response to the first sensed data to generate enhanced simulated performance data for the first deployed product.

[0043] In addition to any one or more of the features described herein, executing the physics-based perception algorithm includes retraining the physics-based perception algorithm based at least in part on the first sensed data associated with the first deployed product.

[0044] In addition to any one or more of the features described herein, the physics-based perception algorithm includes a generative perception algorithm.

[0045] In addition to any one or more of the features described herein, the operations after the user feedback include using the user feedback to train a user model, which includes an electronic model of the user.

[0046] In addition to any one or more of the features described herein, the user model is trained to perform a task including generating user-model-generated feedback including a prediction of feedback that would be generated by the user in response to an instance of the initial cognitive output, wherein the user model includes a digital twin of the user.

[0047] Embodiments of the present disclosure are also directed to computer-implemented methods and computer program products having the above-described features, functions, and combinations of features and functions.

[0048] Further features and advantages are realized through the techniques of the present disclosure. Other embodiments and aspects of the present disclosure are described in detail herein and are considered to be part of the claimed technical concept. For a better understanding of the present disclosure with its advantages and features, please refer to the description and drawings.

[0049] For a more complete understanding of this disclosure, reference should now be made to the following brief description taken in conjunction with the accompanying drawings and detailed description, where like reference characters represent like parts. [Brief description of the drawings]

[0050] [Figure 1] FIG. 1 is a simplified block diagram of a system according to an embodiment of the present disclosure. [Figure 2A] FIG. 1 is a simplified block diagram of a system according to an embodiment of the present disclosure. [Figure 2B] FIG. 1 is a simplified block diagram of a system according to an embodiment of the present disclosure. [Diagram 3] FIG. 1 is a simplified block diagram of a system according to an embodiment of the present disclosure. [Figure 4] FIG. 1 is a simplified block diagram of a system according to an embodiment of the present disclosure. [Diagram 5] FIG. 2 is a flow diagram of a method according to an embodiment of the present disclosure. [Figure 6A] 1 is a simplified block diagram of a system and / or method according to an embodiment of the present disclosure. [Figure 6B] 1 is a simplified block diagram of a system and / or method according to an embodiment of the present disclosure. [Figure 6C] 1 is a simplified block diagram of a system and / or method according to an embodiment of the present disclosure. [Figure 6D] 1 is a simplified block diagram of a system and / or method according to an embodiment of the present disclosure. [Figure 7A] 1 is a simplified block diagram of a system and / or method according to an embodiment of the present disclosure. [Figure 7B] 1 is a simplified block diagram of a system and / or method according to an embodiment of the present disclosure. [Figure 7C] 1 is a simplified block diagram of a system and / or method according to an embodiment of the present disclosure. [Figure 8A] 1 is a simplified block diagram of a system and / or method according to an embodiment of the present disclosure. [Figure 8B] 1 is a simplified block diagram of a system and / or method according to an embodiment of the present disclosure. [Figure 9] 1 illustrates a machine learning system that can be utilized to implement aspects of the present disclosure. [Figure 10] 10 illustrates a learning stage that may be implemented by the machine learning system illustrated in FIG. 9 . [Figure 11]1 illustrates details of an exemplary computing system capable of implementing various aspects of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0051] A detailed description of one or more embodiments of the disclosed apparatus and methods is presented herein, by way of example and not limitation, with reference to the figures.

[0052] Computing resources have evolved in size and efficiency and now support a wide variety of systems and equipment. Computing resources include the hardware and software components of a computer system (e.g., computing system 1100 shown in FIG. 11) that enables problem solving and data analysis. Computing resources can be physical resources, such as servers in a data center, or virtual resources, such as desktops created on demand. Computing resources can also include any software or device that can be accessed by a computer system.

[0053] Computer-based networking technologies enhance the features of a product or system. Relatively small mobile computing processors, sensors, software, and connectivity enable the collection, processing, and transmission of sensed operational data about the product to external systems. Data collected from connected products can be analyzed and used to inform decision-making and enable operational efficiency of the product. Systems and equipment that attempt to function without currently available computing resources may be at a disadvantage.

[0054] Cognitive algorithms are important computing resources for systems, products, and equipment used in various missions. In embodiments of the present disclosure, cognitive systems or cognitive algorithms refer to various algorithm / system types that generate and apply computerized models to simulate human thought processes in complex situations where responses may be ambiguous and uncertain. Traditional cognitive algorithms / systems include self-learning techniques that use data mining, pattern recognition, natural language processing (NLP), and other related techniques to generate mathematical models that make decisions (e.g., classification, prediction, etc.) that substantially mimic human intelligence. In embodiments of the present disclosure, the modifier "cognitive" applied to "output" and / or "output action" refers to the output, action, etc. generated by the cognitive algorithm / system to represent the results of the analytical operations performed by the cognitive algorithm / system.

[0055] A known type of cognitive algorithm is an artificial intelligence (AI) algorithm. In its simplest form, AI is a field that combines computer science with robust data sets to enable problem solving. AI-based computing systems can be implemented as AI models, which are algorithms or computer programs trained on a set of data to recognize certain patterns or make certain decisions without further human intervention. AI encompasses the subfields of machine learning and deep learning. Machine learning utilizes statistical and geometric methods to learn some objective function, while deep learning neural networks (NNs) include an input layer, a hidden layer, and an output layer.

[0056] Although current computing resources are very useful, it can be difficult to effectively deploy the computing resources required for a given mission. In some situations, interaction between the computing resources and internal domain experts is required to enable the computing resources to function properly. When the associated systems, products, and equipment being monitored are controlled by computing resources and are deployed in remote, dangerous, or difficult to access locations (e.g., underwater, in remote areas, in the mountains, in densely forested areas, etc.), it can be difficult to provide sufficient power and / or internal domain expert interaction for such computing resources.

[0057] As previously mentioned, the term "mission" is used herein to refer to any activity having the goal of generating telemetry (i.e., telemetry data) in the process of accomplishing some intended goal. An example of such a mission is a space-related mission. As previously mentioned herein, commercial space endeavors are expanding rapidly. The computing systems, mechanical systems, and know-how required to safely execute and control mission activities in general, and spaceflight activities in particular, are extensive and complex. The growth of all kinds of commercial space endeavors in particular will place a strain on the existing computing systems, mechanical systems, and know-how required to safely execute and control flight and spaceflight activities in general. Such existing mission / flight control systems suffer from a number of shortcomings, including, but not limited to, difficulties in performing efficient and effective trend and performance analysis due, at least in part, to heterogeneous data domains (e.g., different sets of related data collected, formatted and housed in various segregated repositories); manual data extraction (leading to delayed, non-timely analysis); manual data analysis (introducing opportunities for human error in calculations or interpretation); insufficient human capital in data engineering / analytics / science fields relative to the volume of mission-critical needs; and loss of knowledge due to a long-term attrition of internal domain expertise responsible for operating and maintaining the systems.

[0058] Exemplary embodiments of the present disclosure address the above-discussed deficiencies and other deficiencies by providing a novel cognitive learning-based mission analysis and control system that is operable to automatically monitor, analyze, and / or predict data end-to-end to detect trends and problem-solve, and further operable to incorporate and capture human expertise, which can be used for a variety of purposes, including but not limited to extending self-learning capabilities. More specifically, embodiments of the present disclosure provide architectures and methods that include interactions between telemetry generation systems (e.g., physics-based models, deployed hardware, etc.), data storage / repository systems, algorithms, and AI. In some embodiments of the present disclosure, selected AI algorithms and other hardware monitoring / support systems can reside within a digital ecosystem. The architectures and methods further include interactions between computing systems that access the digital ecosystem to present man-machine interfaces (computer screens, audio, etc.), and humans for health monitoring (prognosis, anomalies, performance, etc.) and management of the mission hardware. The system captures and retains data generated and created by the mission hardware and human internal domain experts. The collected data is used to create digital twins of both the hardware and humans that interact with the system through the use of artificial intelligence and other algorithms. As new data is created, the system autonomously self-learns and updates the digital twins, algorithms, and artificial intelligence so that the virtual representations are up-to-date. Non-limiting examples of these features are illustrated in Figures 1, 2A, and 2B and described in further detail later in this specification.

[0059] In addition to any one or more of the features described herein, the disclosed systems and methods further enable knowledge capture and retention, such that humans (e.g., internal subject matter experts (SMEs)) provide context to the stored telemetry. Non-limiting examples of these features (human-in-the-loop validation, data labeling, and crowd-sourced training) are illustrated in Figures 3 and 5 and described in further detail later herein.

[0060] In addition to any one or more of the features described herein, the disclosed systems and methods further include AI and other algorithms operable to create a virtual replica (e.g., a digital twin) of the SME. In some embodiments of the present disclosure, the above human / SME knowledge capture and retention can be combined into a knowledge base and used to create a comprehensive composite virtual SME. In some embodiments of the present disclosure, the virtual SME is interacted with in the form of a chat robot, a NL interface, or a graphical user interface (GUI). Non-limiting examples of these features are illustrated in FIG. 4 and described in further detail later in the specification.

[0061] In addition to any one or more of the features described herein, the disclosed systems and methods further include using the aforementioned digital twin to generate synthetic data of unknown failure modes. In embodiments of the present disclosure, a digital twin of the hardware being monitored is used to generate synthetic data of specific events that are unobservable or difficult to observe in actual telemetry. In some embodiments of the present disclosure, the synthetic data set is used to train AI and other algorithms to detect, predict, and / or understand unknown or rare events. Non-limiting examples of these features are illustrated in FIGS. 6A-6D and described in further detail later herein.

[0062] In addition to any one or more of the features described herein, the disclosed systems and methods include an AI engine process that provides multiple paths to a consumer (e.g., Consumer 701 shown in FIG. 7A ) to take in information from either the PHM AI engine metamodel (e.g., PHM AI engine metamodel 706 shown in FIG. 7A ) or the flight hardware (e.g., Flight Hardware 707). Upon receiving this information, the consumer either ultimately consumes the information, i.e., does not return further information to the AI ​​engine process, or the consumer can interact / pass further data through the system. If the consumer desires / is commanded to return information through the AI ​​engine process, there are at least two channels through which the data can be sent. First, the data can be sent back as regular telemetry (e.g., Telemetry 702 shown in FIG. 7A ), i.e., the data can be sent back to a telemetry and data repository (e.g., Telemetry and Data Repository 704 shown in FIG. 7A ) for archival purposes but with the intent to be migrated back, not for immediate action. Second, data can be sent back as triggered event telemetry (e.g., triggered event telemetry 703), i.e., with the intent that the data be migrated back to the telemetry and data repository with the hope that immediate action will be taken. When data flowed through the AI ​​Engine process is sent through a trigger event channel, the PHM AI Engine metamodel uses the trigger and the new data to automatically initiate some task event (such as updating / retraining an existing model, completely rebuilding an existing model, performing model prediction drift detection, etc.). Non-limiting examples of these features are illustrated in FIG. 7A and described in more detail later in this specification.

[0063] In addition to any one or more of the features described herein, the disclosed systems and methods further include a novel approach to sensors (e.g., inferential sensors 715 shown in FIG. 7B), referred to herein as "synthetic sensors." For example, by using digital twins as described herein, everything in a system can be measured and observed, whereas in a real system, the number of sensors and measurements is often constrained. Through the use of AI and algorithms, embodiments of the present disclosure allow various measurements to be learned as they relate to real sensors on the hardware (e.g., using simultaneous / inferential synthetic sensors 715 shown in FIG. 7B), thereby determining the value of "virtual" measurements on the real system when no sensors are present to actually observe the measurements. In some embodiments of the present disclosure, the disclosed synthetic / inferential sensors can be used to generate additional data that can be used to train the system's AI and other algorithms, and to perform diagnostics on the system hardware (e.g., deployed product 142 shown in FIG. 1). Non-limiting examples of these features are illustrated in FIG. 7B and described in more detail later herein.

[0064] In addition to any one or more of the features described herein, the disclosed systems and methods further include a novel use of the disclosed digital twin to improve data sampling frequency. In some embodiments of the present invention, the disclosed digital twin is used to improve sampling frequency by essentially filling gaps in the data set of hardware that produces data at a low sampling rate. For example, if a given hardware device provides one data point per minute, embodiments of the present disclosure provide a model that provides accurate data points associated with the given hardware device at one per second. Non-limiting examples of these features are illustrated in FIG. 7C and described in further detail later in this specification.

[0065] In addition to any one or more of the features described herein, the disclosed systems and methods further include novel self-calibrating digital twins, where the self-learning elements of the disclosed systems and methods use AI and / or other algorithms to automatically recalibrate the digital twin(s) based on incoming telemetry, thereby synchronizing the digital twin to be representative of the actual hardware. Non-limiting examples of these features are illustrated in FIG. 8A and described in more detail later herein.

[0066] In addition to any one or more of the features described herein, the disclosed systems and methods further include new scheduled and event-triggered retraining / calibration of digital twins and AI models. These features enable the disclosed systems and methods to automatically update associated algorithms / models in a digital ecosystem (e.g., digital ecosystem 120 shown in FIG. 1) including digital twins and PHM-related AI algorithms based on scheduled, human-triggered, or machine-triggered events. Non-limiting examples of these features are illustrated in FIG. 8B and described in further detail later herein.

[0067] FIG. 1 illustrates a novel cognitive learning-based mission analysis and control system 100 according to an aspect of the present disclosure. The system 100 is operable to support various missions or exploration types, including land-based, underground, water-based, Earth atmosphere-based, and space-based locomotion, for various purposes, including, for example, military operations, defense operations, and / or learning about such domains. Such missions / explorations range from land with any kind of terrain (e.g., mountains, caves, etc.), underground at various depths, water at various depths, air within the Earth atmosphere, and beyond the Earth atmosphere into space. The architecture of the system 100 is operable to utilize the computational resources of the remote station 150 in that some or all of the computational resources and architecture of the system 100 can be located at the remote station 150, thereby providing computational resources to various components without the constraints of limited computational power that may be available for the mission / exploration. The self-learning capabilities of the system 100 allow for scaling of knowledge collection without relying on limited data science and software engineering resources.

[0068] System 100 includes a data repository 110, a digital ecosystem 120, and a user interface 130, configured and arranged as shown. In general, a digital ecosystem is a set of technologies working together as a network / unit of digital devices to provide information about the entire system. Digital ecosystem 120 includes various computer-based monitoring and control systems configured and arranged to perform cognitive learning based mission analysis and control actions that automatically monitor and analyze sensed operational data to detect trends and perform problem solving related to various deployed products 142 and other mission / exploration systems associated with a given mission / exploration. The computer-based monitoring and control systems of digital ecosystem 120 include AI systems of various configurations and types.

[0069] The data repository 110 is configured and arranged to automatically ingest various data types (e.g., sensed operational data) from various data sources including the physics-based modeling engine 140, the deployed products 142 (via the sensor network 144), the remote stations 150 (via the antenna network 146), and at least one internal domain expert (SME) 402. In an embodiment of the present disclosure, the digital ecosystem 120 includes components and functions operable to automatically display messages, queries, and other prompts to the SME 402 to ensure that the SME 402 inputs SME knowledge including both SME decisions and SME rationale for the SME decisions. In some embodiments of the present disclosure, the data repository 110 can provide data to the physics-based modeling engine 140 trained to generate synthetic data of system behavior, particularly unknown / infrequent fault events and unknown / non-occurring machine states, which can also be stored in the data repository 110. Synthetic data is information that is artificially manufactured rather than generated by real-world events. It is algorithmically created and used as a surrogate for test data sets of production or operational data to validate mathematical models of digital ecosystem 120 and to train machine learning (ML) models of digital ecosystem 120. Physics-based (or physics-based) modeling engine(s) 140 are modeling engines that are constructed and arranged to include physics-based constraints (e.g., governing equations that describe the physics of the relevant phenomenon that is the subject of the physics-based modeling engine). These equations represent additional training information regarding the relationships between input parameters to the model and the model's output(s).

[0070] The interface 130 of the system 100 can be implemented in a variety of user interface (UI) configurations. The interface 130 of the system 100 is configured and arranged to consume insights generated through the digital ecosystem 120 and provide human-in-the-loop feedback to the self-learning capabilities of the system 100. The interface 130 of the system 100 can include, for example, a digital assistant operable to provide real-time query response (voice-to-voice, text-to-voice, voice-to-text, or text-to-text) or request-to-action agent located at the edge (i.e., at the point of use by the user). The digital assistant can include AI-based functionality that can query any information within the digital ecosystem 120 (and potentially information outside the digital ecosystem 120) via voice or text prompts and receive a corresponding response. Additionally, a user of the AI-based digital assistant can perform an action in response to a request (e.g., a request to log that some event has occurred in the data repository 110). Effectively, the AI-based digital assistant provides specialized functionality related to the mission / exploration data and actions. The capabilities of the AI-based digital assistant can also be accessed and used by support personnel (e.g., mission control) at the remote station 150, as well as by SMEs 402, to query information and provide feedback to the digital ecosystem 120.

[0071] 2A illustrates a novel cognitive learning-based mission analysis and control system 100A according to an aspect of the present disclosure. System 100A is a non-limiting example of how system 100 (shown in FIG. 1) may be implemented in an application where the mission / exploration is a space-based mission / exploration, specifically including a space-based mission / exploration that uses computing resources and human personnel to monitor space mission hardware for both human and non-human spacecraft. In system 100A, data repository 110 (shown in FIG. 1) is implemented as data repository 110A, digital ecosystem 120 (shown in FIG. 1) is implemented as digital ecosystem 120A, and user interface 130 (shown in FIG. 1) is implemented as user interface 140A.

[0072] The system 100A is operable to support space exploration with a focus on optimizing the operation of flight systems and / or deployed products 142A, such as, but not limited to, the Environmental Control and Life Support System (ECLSS). The ECLSS is a system of regenerative life support hardware that serves all functions (e.g., air revitalization, temperature control, humidity control, biological waste management, water generation, etc.) necessary to sustain and preserve life present on any spacecraft (e.g., space station, space suit, space vehicle, etc.) or space habitat. The creation of the ECLSS allows for more personnel to be accommodated in space, extends the time that personnel can remain in space, and significantly reduces overall operating costs by recycling resources. The ECLSS aboard the International Space Station (ISS) includes two main components, the Water Recovery System (WRS) and the Oxygen Generation System (OGS). The WRS provides clean water by recirculating crew urine, cabin humidity condensate, and EVA (Extra Vehicle Activity) waste. Reclaimed water must meet strict purity standards before it can be utilized to support crew, laboratory, EVA, and payload activities. The WRS includes the UPA (Urine Treatment Assembly) and WPA (Water Treatment Assembly). The OGS generates oxygen for air breathing and also replaces oxygen lost as a result of experiment use, airlock depressurization, module leaks, and carbon dioxide emissions. The OGS includes the OGA (Oxygen Generation Assembly) and PSM (Power Supply Module). Oxygen is generated at selectable rates and can operate continuously and cyclically.

[0073] System 100A is a near-Earth hardware that can utilize the Earth's computing resources in that some or all of the computing resources of system 100A can be located on Earth, thereby providing computing resources to spaceflight hardware without being constrained by the limited computing power onboard existing products. In general, near-Earth refers to the orbital region of space from Earth to the Earth's Moon, including low Earth orbit, medium Earth orbit, medium Earth orbit, and high Earth orbit. In an embodiment of the present disclosure, the computing resources on Earth can be implemented as remote stations 150 (shown in FIG. 1 ) that communicate with antenna network 152A. The self-learning capabilities of system 100A enable scaling of knowledge collections without relying on limited data science and software engineering resources.

[0074] 2A, system 100A includes a data repository 110A, a digital ecosystem 120B, and a user interface 130A, as configured and arranged as shown. Data repository 110A (e.g., a SQL server) of system 100A is configured and arranged to automatically populate flight telemetry and internal domain expert (SME) knowledge (e.g., from deployed products 142A via distributed sensor network 144, which may include smart sensors with on-board processor capabilities). In an embodiment of the present disclosure, system 100A includes components and functionality (e.g., digital ecosystem 120C shown in FIG. 3) operable to automatically display messages, queries, and other prompts to SMEs 402 to ensure that the SMEs input their knowledge into system 100A, including both the SME's decisions and the SME's rationale for the SME's decisions. In some embodiments of the present disclosure, the telemetry can be provided to one or more physics-based modeling engines 140A trained to generate synthetic data of system behavior, particularly unknown / infrequent fault events and unknown / non-occurring machine states, which can also be stored in the data repository 110A. Synthetic data is artificially manufactured information, rather than generated by real-world events. It is algorithmically created and used as a substitute for test data sets of production or operational data to validate mathematical models and train machine learning (ML) models. The physics-based (or physics-based) modeling engine(s) 140A are modeling engines that are configured and arranged to include physics-based constraints (e.g., governing equations that describe the physics of the relevant phenomenon that is the subject of the physics-based modeling engine 140A). These equations represent additional training information regarding the relationship between the input parameters to the model and the output(s) of the model.

[0075] In some embodiments of the present disclosure, synthetic data of unknown / occasional fault events and machine conditions, as well as product telemetry, are consumed by an artificial intelligence (AI) engine for the purpose of rapidly generating prognostic and health management (PHM) models of the underlying operating hardware. The PHM AI engine may be part of the digital ecosystem 120A and leverages capabilities of classical statistics, traditional statistics and machine learning, deep learning, reinforcement learning, and the like. The PHM AI engine is configured and arranged to provide real-time and post-mortem data analytics, including but not limited to risk assessment, early fault diagnosis, system health prediction, sensor health prediction, and maintenance management in the system 100A. The PHM AI engine implements some or all of the AI ​​prognostic capabilities of the digital ecosystem 120A to identify machine health status, alert users (e.g., SMEs, system maintenance personnel, etc.) to system degradation, predict when future maintenance will be required, and suggest optimal maintenance methods. Maintenance work can be scheduled, thereby avoiding unplanned downtime. The PHM AI engine also facilitates advanced planning for labor and spare part management based on maintenance plans, thereby reducing repair times and improving the overall efficiency of how repair resources are used.

[0076] In some embodiments of the present disclosure, the data repository 110A can be implemented as a searchable database operable to organize and store data from various sources within a segment or area of ​​the data repository. The data repository 110A shown in FIG. 2A can be any type of database, including but not limited to a relational SQL database, a noSQL unstructured database, an unstructured data lake, a time series database, etc., or any combination thereof. In some embodiments of the present disclosure, the data repository 110A can include features and functionality of a relational database operably controlled by a computing system 1100 (shown in FIG. 11). In general, a database is a means of storing information so that it can be retrieved from it, and a relational database presents information in tables with rows and columns. A table is called a relational table in the sense that it is a collection of objects of the same type (rows). Data in a table can be related according to a common key or concept, and the ability to retrieve related data from a table is the basis of the term relational database. A database management system (DBMS) of the computing system 1100 controls how data in the data repository is stored, maintained, and retrieved. The database management system of computing system 1100 performs the task of determining how data and other information (e.g., natural language text descriptions of the predictive feedback provided by SMEs) is stored, maintained, and retrieved from data repositories.

[0077] The sensor network 144 is a distributed sensor network having one or more sensor components coupled to some or all of the components of the deployed products 142A of the system 100A, and generates date and information regarding various operations performed by the deployed products 142A in the system 100A. The widespread availability and relatively low cost of miniaturized computing systems has greatly increased the ability of distributed sensor networks (e.g., the sensor network 144) to collect electronic information and / or data regarding any activity of the system 100A that can be monitored and stored using technology. The collected electronic information / data is generally referred to as raw information / data, and is captured / stored in various information formats. Generally, raw data is data that has not yet been processed, coded, formatted, or analyzed for useful insights. In other words, raw data is data that has been collected from one or more sources but is still in its initial, unaltered state. In some embodiments of the present disclosure, the raw data / information collected from system 100A by distributed sensor network 144 is stored in a data repository in raw form and downstream data pre-processing techniques are used to prepare the data / information for analysis by various data analysis components (e.g., components of digital ecosystem 120A). In some embodiments of the present disclosure, the pre-processing techniques are applied by digital ecosystem 120A before or immediately after the raw data / information collected from system 100A by distributed sensor network is stored in the data repository.

[0078] The digital ecosystem of system 100A is operable to use data and / or information (including both structured and unstructured data) generated by various sources to generate insights regarding product performance via analysis, prognosis, anomaly detection, probabilistic event explanation, etc. Embodiments of the present disclosure use various cognitive computing systems, including but not limited to artificial intelligence (AI), algorithms, models, code libraries, logic, rules, etc., to analyze the data and / or information to generate insights regarding product performance via analysis, prognosis, anomaly detection, probabilistic event explanation, etc. Examples of basic features and functionality of computing systems and AI algorithms that can be used to implement aspects of the present disclosure are illustrated in FIGS. 9-11 and described in further detail subsequently herein.

[0079] Because the types of data / information analyzed by the digital ecosystem are diverse, the cognitive computing systems of the digital ecosystem include capabilities to ingest and analyze diverse types of data / information, including but not limited to structured data, text documents, images, recorded audio, chat logs, etc. For example, the cognitive computing systems of the digital ecosystem include natural language processing (NLP) capabilities. NLP is a field of computer science that uses algorithms and computer systems to process human languages, such as English. Human languages ​​are often referred to as natural languages. In general, the term "natural languages" refers to languages ​​that have been developed over time by humans as a way of communicating between people, rather than languages ​​that have been created for communication between non-human entities such as computers.

[0080] NLP is used in systems that allow humans to interface more effectively with data repositories that store electronic information. NLP interfaces / systems have been developed to perform various human / data interface tasks, such as text search and / or text matching, as well as more advanced tasks, such as document / data content analysis (DCA). In general, DCA systems use categorization and classification of speech, written text, dialogue, images, or other forms of electronically stored information to perform computer-aided exploration and analysis. A known type of DCA is the so-called "Question and Answer (QA) system", which uses NLP, machine learning algorithms, and various language models (e.g., Large Language Models (LLMs)) to cognitively analyze accumulated information from various sources in order to provide answers to open-ended natural language questions.

[0081] In some embodiments of the present disclosure, a "question and answer" (QA) system (e.g., as shown in FIG. 3) is used in a novel way to facilitate SMEs to insert useful and complete information into the digital ecosystem and / or data repository. The resulting information / data extracted via the QA system is then consumed in training by an AI engine and integrated into the overall data used to develop further analytics. For example, the QA system of the digital ecosystem can be trained to assess the sufficiency of the SME's predictive feedback provided in the form of NL text, and then the SME can formulate and present follow-up requests / questions to the SME targeted to elicit additional information that would make the SME NL predictive feedback more meaningful. For example, if the digital ecosystem generates a prediction that Valve A is showing signs of breaking in 2 months and should be replaced / repaired, the SME can override this prediction and provide the digital ecosystem with an SME NL predictive feedback explanation that reads, "Valve A does not need to be replaced for at least another 12 months; set reassessment of Valve A for 10 months from now." The QA system may evaluate the sufficiency of this SME's explanation for the NL prediction feedback, determine that it is insufficient in that it does not provide any details as to why the SME reached this conclusion, and create an NL request asking the SME to submit an explanation as to why the SME reached a conclusion about when Valve A will need repair. For example, the SME may have learned from attending a recent seminar that the best predictor of Valve A's failure is performance parameter A. The SME queries the digital ecosystem to provide the current value of parameter A, references a set of guidelines the SME obtained in the seminar that converts the current value of parameter A to a future repair time, and determines from that source that Valve A will maintain its performance for at least the next 12 months. In response to the NL request from the QA system, the SME may provide some or all of the above rationale to the QA system.The QA system may be further configured or trained to evaluate the sufficiency of the response to the request for additional information and continue to pose follow-up questions until the QA system determines that a sufficient response has been received. In an embodiment of the present disclosure, the QA system may be configured to measure sufficiency based on the criteria of "what," "why," and "source." The "what" criteria evaluate whether the SME is clear about what they want for their action or inaction, the "why" criteria evaluate whether they provide a proper rationale for the "what" portion of the SME's answer, and the "source" criteria evaluate whether they provide a source for the "what" and "why" portions of the SME's answer. Additional details of how the system 100A interacts with SMEs, according to an embodiment of the present disclosure, are shown in FIGS. 3-5 and described in more detail herein below.

[0082] Cognitive computing systems in the digital ecosystem can include general data science capabilities and anomaly detection capabilities, including but not limited to mathematics, statistics, specialized programming, advanced analytics, artificial intelligence (AI), and machine learning (ML), as well as specific subject matter expertise aimed at uncovering actionable insights hidden in data, predicting future performance / events, identifying hidden patterns and relationships, creating autonomous decision systems, etc. The insights can be used to guide decision making and strategic planning. A key consideration in data science is the quality of the data being analyzed. Data quality can be affected by so-called "anomalous" or "outlier" data. The term "anomaly" refers to a data point or set of data points that deviates dramatically from the expected sample and pattern of those types. For a data set that follows a standard bell curve, anomalies are the far right and far left data. Anomalies can indicate fraud or some other abnormality, but can also be measurement errors, experimental problems, or one-off instances of novelty. Anomalies and / or outliers can hinder data analysis techniques and skew the analysis results.

[0083] Anomaly detection is the process of detecting anomalous behavior of a system or device with the goal of diagnosing the cause of the anomalous behavior and possibly taking corrective action to address the anomaly. In an embodiment of the present disclosure, the anomaly detection function determines when hardware monitored by the system 100A exhibits anomalous behavior based on received data indicative of the hardware performance. The purpose of the anomaly detection function is to ensure that hardware issues are detected by an anomaly signature(s) in the data. As a non-limiting example, in the case of the image domain, an anomaly detection process can be used in quality control operations to inspect surfaces to identify unacceptable defects or imperfections in the associated product. Anomaly detection uses mathematical techniques to detect anomalies in a dataset (e.g., a dataset representing an image) based on how different a given data point is from its surrounding data points or from a standard deviation. Anomaly detection tasks can be performed by neural networks using deep learning algorithms. Some deep learning algorithms require large amounts of labeled (annotated) data to learn so-called "deep features" in order to train effective models for performing cognitive operations such as prediction, classification, etc. However, in many anomaly detection deep learning applications, labeled anomaly training data is unavailable or insufficient in quantity due to various factors. Therefore, so-called "zero-shot" learning techniques have been developed to train machine learning algorithms to perform classification and / or prediction tasks when the machine learning algorithm has never seen or been trained on examples of the actual classification / prediction task before. In other words, zero-shot learning may enable a machine algorithm to perform a classification / prediction task where examples of actual to-be-predicted (TBP) data are "unknown" to the machine learning algorithm(s). In zero-shot learning, the classes covered by the training instances and the classes that the classification / prediction task needs to classify are separate.Thus, zero-shot learning techniques are designed to overcome the lack of training examples in classification / prediction tasks by leveraging details learned from training examples of tasks related to, but distinct from, the target classification / prediction task. Since both training classes and unknown task classes are related to a high-dimensional vector space called a semantic space, the details learned from training examples of related / distinct tasks are used to draw inferences about the unknown classes of the target classification / prediction task. Thus, known zero-shot learning techniques can include a training phase and an inference phase. In the training phase, knowledge about attributes of the intermediate semantic layer is captured. In the inference phase, this knowledge is used to classify instances among a set of new classes.

[0084] Another machine learning technique for performing zero-shot learning is known as "transfer learning". Transfer learning is a machine learning method in which a model developed for a first task is reused as a starting point for a model on a second, different but related task. For example, in deep learning applications, pre-trained models are used as starting points for various computer vision and natural language processing tasks. Transfer learning leverages the extensive knowledge, skill, computational, and time resources required to develop neural network models by reusing them. In an attempt to avoid the significant time it takes to develop labeled training data to train an anomaly detection model to perform an anomaly detection task in a target domain, transfer learning techniques have been developed that leverage training data from different but related domains. The domain of the TBP data is called the target domain (TD) and the domain of the different but related task is called the source domain (SD).

[0085] Embodiments of the present disclosure use transfer learning to scale knowledge across products and platforms of systems 100, 100A, 100B (shown in FIGS. 1, 2A, 2B). For example, a first cognitive algorithm model of system 100A can transfer its learning to a second cognitive algorithm model of the system, where the first and second cognitive algorithm models are trained to perform a task that has some overlapping areas and some non-overlapping areas. As another example, a learned model of the functionality of a water treatment device can be applied via transfer learning to another similar water treatment system on a different spacecraft.

[0086] The cognitive computing system of the digital ecosystem may further include general hardware performance analysis applied to one or more deployed products 142A (e.g., ORUs (orbital replaceable units)). Such analysis refers to, but is not limited to, calculating and / or capturing both explicit and implicit information in data related to hardware performance. Such analysis may include univariate and multivariate statistics of sensors embedded in the hardware, calculated metrics that cannot be explicitly observed in telemetry, and analysis of relative / relevant data in the hardware. Returning to the ECLSS example, in the case of the water generation component of life support, such general hardware performance may include statistical measures (mean, variance, min, max, skew, etc.) of all sensors on the system, the amount of water produced in a given time and its production rate, the amount of urine distilled back into the water supply, the number of times a particular pump was activated for the purpose of purifying water or maintaining system pressure, correlations between various system components, slopes / trends / patterns / other behavioral information regarding the sensors and subsystems in the system, etc.

[0087] The user interface(s) (UI) 130A of the system 100A is configured and arranged to consume digital ecosystem insights and provide human-in-the-loop feedback to the self-learning capabilities of the system 100A. The UI 130A of the system 100A can include, for example, a graphical user interface (GUI) (e.g., interactive WebApp, dashboard, mobile app, etc.), on-board hardware (e.g., ECLSS hardware, flight computers, space creature avionics, space suit heads-up displays, etc.), and digital assistants (e.g., NL interfaces such as the QA system shown in FIG. 3 and the on-edge AI copilot shown in FIG. 2A). The on-edge AI copilot described in the embodiments of the present disclosure refers to a real-time query response (voice-to-voice, text-to-voice, voice-to-text, or text-to-text) or action request agent located at the edge (i.e., a point of use by the user such as in a space suit helmet). A user of the AI ​​Copilot (e.g., SME 402) can query any information within the digital ecosystem 120A (and potentially information outside the digital ecosystem) via voice or text prompts and receive corresponding responses. Additionally, a user of the AI ​​Copilot can request that the Copilot perform an action (e.g., a request to log that some event has occurred in a data repository). Effectively, the AI ​​Copilot provides specialized capabilities related to mission data and actions. The AI ​​Copilot functionality can also be accessed and used by Earth-based support personnel (e.g., mission control at the remote station 150 shown in FIG. 1) and SMEs to query information and provide feedback to the digital ecosystem 120A.

[0088] FIG. 2B illustrates an exemplary application of system 100A (shown in FIG. 2A), where the system illustrated in FIG. 2B is designated as system 100B having substantially the same features and functionality as system 100A. Referring now to a description of the operational steps of an exemplary application of system 100B, telemetry from the ISS WPA (International Space Station Water Processing Assembly) is streamed and collected in data repository 110A. Scripts in system 100B access the collected data to run prognostic models, anomaly detection algorithms, and calculate WPA performance metrics (e.g., daily water production) on newly acquired telemetry. In an embodiment of the present disclosure, prognostic and anomaly detection AI models can be rapidly developed using the AI ​​generation engine (e.g., PHM AI engine) of system 100B. In an embodiment of the present disclosure, events that have not yet occurred or are rarely observed in the data are simulated using the previously described physics-based (or physics-based) modeling engine 140A. SMEs access system 100B via an interactive dashboard that recommends preventative maintenance actions, reports anomalies to investigate, and displays WPA performance metrics. SME 402 uses this information to assess the health of the WPA and make decisions regarding the operation of the WPA (e.g., deciding to change multiple filtration beds). Actions and events (e.g., maintenance events) are logged within system 100B, and these events are used to automatically update and retrain the prognostic and anomaly detection models of digital ecosystem 120A. Knowledge of the events is stored with the data, thereby providing context for future and / or related personnel, and maintaining a complete log and source of system-generated and SME-generated knowledge encompassing the product history of the WPA.

[0089] FIG. 3 shows additional details regarding how the SME 402 interacts with the systems 100, 100A, 100B. In FIG. 3, the digital ecosystem 120C of the system 100B is illustrated, showing AI prognostics, AI anomaly detection, WPA performance metrics, NLP, and other learning-based functions, configured and arranged as shown. FIG. 3 further shows the system prediction functions 330 generated by the digital ecosystem 120C. A QA system 320 in electronic communication with the digital ecosystem 120C and the SME model 310 is configured and arranged as shown. Although the system predictions 330 are referenced in FIGS. 3, 4, and 5, it is understood that the examples illustrated in FIGS. 3, 4, and 5 and described herein apply to a more general example in which the system predictions are any type of output from the various analytical and cognitive computing systems of the systems 100, 100A, 100B. According to embodiments of the present disclosure, the SME model 310 can be implemented in a variety of ways. In the example shown in Figure 3, the SME model 310 is implemented using an ML algorithm 912 and an NLP algorithm 914 to generate one or more models 916 that function as the SME model 310. Details of how the SME model 310 may be implemented are shown in Figures 9 and 10 and are described in more detail herein below.

[0090] In operation, the SME 402 may be one or more individuals who are tasked with using their subject matter expertise to evaluate system predictions 330 generated by the system 100, 100A, 100B. Although depicted as system predictions 330, embodiments of the present disclosure apply to various analysis-related outputs from the system 100, 100A, 100B (e.g., anomaly detection, prognosis, performance metrics, other metrics that can be used to evaluate how well a piece of equipment is functioning, etc.) that may or may not be actual predictions. In some embodiments of the present disclosure, the system 100, 100A, 100B requires input from the SME 402 before completing and / or implementing the system predictions 330. The SME 402 generates SME predictive feedback in response to the system prediction 330, which may take a variety of forms including, for example, an action (an indication that the SME reviewed and approved / rejected / modified the system prediction 330) and / or an explanation (e.g., NL text specifying the actual approval / rejection / modification of the system prediction 330, the rationale behind the approval / rejection / modification, and the sources the SME 402 used in formulating the rationale). The SME predictive feedback is provided to the digital ecosystem 120C and the SME model 310. The SME model 310 uses the SME predictive feedback and the system prediction to train the SME model 310, and in effect provides SME model predictive feedback that tracks the actual SME predictive feedback generated by the SME 402. The SME model 310 provides a mechanism to capture the knowledge of the SME 402 by capturing the interactions the SME 402 has with the systems 100, 100A, 100B and training the SME model 310 to generate SME model feedback in response to new system predictions 330 that mimics the SME predicted feedback that would have been provided by the SME 402. The SME model 310 can be used in a variety of situations including, for example, when the SME 402 is unavailable for a period of time (e.g., a week's vacation) and the SME model 310 acts in place of the absent SME 402.

[0091] Digital ecosystem 120C receives SME prediction feedback and provides it to QA system 320. QA system 320 is used in a novel way to facilitate SMEs 402 to insert useful and complete information (e.g., via a graphical user interface (GUI)) into various cognitive models of digital ecosystem 120C and / or into data repositories 110, 110A. In some embodiments of the present disclosure, the information inserted by SMEs 402 into QA system 320 can be inserted into models of digital ecosystem 120C as updated training data and used to generate updated model outputs (e.g., updated model predictions) of digital ecosystem 120C. In some embodiments of the present disclosure, the information inserted by SMEs 402 into QA system 320 can be accumulated and used by NLP of digital ecosystem 120C to generate and update the knowledge corpus of SMEs 402 associated with systems 100, 100A, 100B.

[0092] In some embodiments of the present disclosure, a cognitive computing module of the QA system 320 of the digital ecosystem 120C can be trained to evaluate the sufficiency of SME predictive feedback provided in the form of natural language (NL) text, speech, etc., and to create and present follow-up requests / questions to the SME 402 targeted at eliciting additional information from the SME 402 that would make the SME NL predictive feedback more meaningful. For example, if the digital ecosystem 120C generates a prediction that Valve A is showing signs of breaking in 2 months and should be replaced / repaired, the SME 402 can override this prediction and provide the digital ecosystem 120C with an SME NL predictive feedback explanation of "Valve A does not need to be replaced for at least another 12 months; set reassessment of Valve A for 10 months from now." The QA system 320 may evaluate the sufficiency of this SME's explanation of the NL prediction feedback, determine that it is insufficient in that it does not provide any details as to why the SME 402 came to this conclusion, and create an NL request asking the SME 402 to submit an explanation as to why the SME 402 came to this conclusion as to when valve A will need repair. For example, the SME 402 may have learned from attending a recent seminar that the best predictor of valve A's failure is performance parameter A. The SME 402, without the knowledge of the QA system 320, queries the systems 100, 100A, 100B to provide the current value of parameter A, references a set of guidelines that the SME 402 obtained at the seminar that converts the current value of parameter A to a future repair time, and determines from that source that valve A will maintain its performance for at least the next 12 months. In response to the NL request from the QA system 320, the SME 402 may provide some or all of the above rationale to the QA system 320. The QA system 320 may be further configured or trained to evaluate the sufficiency of the response to the request for additional information and continue to present follow-up questions to the SME 402 until the QA system 320 determines that a sufficient response has been received from the SME 402.In some embodiments of the present disclosure, the QA system 320 can be configured to measure sufficiency in any suitable manner, including, for example, based on the criteria of "what," "why," and "source." The "what" criteria evaluate whether it is clear what the SME 402 wants for their action or omission, the "why" criteria evaluate whether an appropriate rationale is provided for the "what" portion of the SME's answer, and the "source" criteria evaluate whether the SME 402 provides a source for the "what" and "why" portions of the SME's answer. Additional details of how the system 100, 100A, 100B interacts with the SME 402, according to embodiments of the present disclosure, are illustrated by the method 500 illustrated in FIG. 5 and described in more detail herein below.

[0093] In some embodiments of the present disclosure, the SME model 310 shown in FIG. 3 can be implemented by generating a digital twin of the SME 402. FIG. 4 shows additional details of how an SME digital twin 402A of the SME 402 can be generated using a digitization module 420 and a digital twin processor module 410, according to aspects of the present disclosure. As shown, the digitization module 420 digitizes various data types reflecting the SME's internal knowledge "domain" including, for example, SME profile data / information 412, SME background knowledge corpus 414 (e.g., relevant publications written or read by the SME 402), SME predictive feedback actions 416, all historical SME email traffic, and SME predictive feedback explanations 418. The digitization module 420 provides the digitized data to the digital twin processor 410, which uses known machine learning and NLP techniques (e.g., large language models (LLMs)) to create and update the user digital twin 402A. Once trained, the SME digital twin 402A can be used in substantially the same manner as the SME model 310, with the added benefit that new hires can subsequently interact with the digital twin 402A for educational and outreach purposes. Both the SME model 310 and the SME digital twin 402A can include and utilize NLP / LLM functionality that allows for chatting and execution with the SME model 310 and / or the SME digital twin 402A.

[0094] The operation of the system 100, 100A, 100B will now be described primarily with reference to the method 500 shown in FIG. 5, with appropriate reference to corresponding elements, steps, and diagrams in FIGS. 1-4B. The method 500 begins at block 502 and then proceeds to block 504, where the system 100, 100A, 100B accesses a first or next system prediction (SP) (e.g., system prediction 330). The method 500 proceeds to decision block 506 to determine whether SME feedback regarding the first or next SP has been received by the system 100, 100A, 100B. If the response to the inquiry at decision block 506 is no, the method 500 moves along path A to block 508, where the results of the analysis performed at decision block 506 and the collected information (including the analysis performed and the collected information as described below) are used in block 508. If the response to the inquiry at decision block 506 is no, the method 500 also proceeds to block 510 to execute the first or next SP to record the absence of SME feedback in the system 100, 100A, 100B (e.g., in a data repository). In some embodiments of the present disclosure, the decision block 506 may operate to identify the absence of SME feedback condition in any suitable manner. In some embodiments of the present disclosure, the decision block 506 may assume the absence of SME feedback after not receiving user feedback within a predetermined time. In some embodiments of the present disclosure, the decision block 506 may request confirmation of (or agreement with) the absence of SME feedback condition from the SME 402 after not receiving user feedback within a predetermined time. In some embodiments of the present disclosure, the decision block 506 may be configured to request confirmation of the absence of SME feedback condition from the SME 402 after not receiving user feedback within a predetermined time, and may be further configured to not proceed further with the method 500 until the requested confirmation of the absence of SME feedback condition is received from the SME 402.In embodiments of the present disclosure where a confirmation of no SME feedback is received, the confirmation of no SME feedback is recorded in the system 100, 100A, 100B (e.g., in the data repository 110, 110A) along with the no SME feedback. In embodiments of the present disclosure, the system 100, 100A, 100B is operable to treat the no SME feedback and / or confirmation of no SME feedback as an agreement of the SME with the first or next SP. In embodiments of the present disclosure, all functionality of the QA system 320 (shown in FIG. 3 ) may be incorporated within decision block 506 to facilitate the above confirmation of no SME feedback and / or evaluate the sufficiency of the confirmation of no SME feedback as described above. From block 510, the method proceeds to decision block 514.

[0095] Returning to decision block 506, if the answer to the inquiry at decision block 506 is yes, the method 500 moves or branches in two directions. The method 500 proceeds to block 508 and uses the results of the analysis performed at decision block 506 and the collected information to train / update the relevant cognitive systems of the digital ecosystem 120, 120A, 120B, 120C, including the SME-model 402 and / or the SME digital twin 402A, and the method 500 also proceeds to block 512 and uses the SME feedback to perform the initial or next SP 330, further update the various system models of the systems 100, 100A, 100B based on the SME feedback, and record the SME feedback in the data repository 110, 110A. In an embodiment of the present disclosure, all the functionality of the QA system 320 (shown in FIG. 3) can be incorporated within decision block 506 to facilitate the generation of the above-mentioned SME feedback and / or evaluate the sufficiency of the above-mentioned SME feedback. From block 512, the method 500 proceeds to decision block 514.

[0096] At decision block 514, the method 500 evaluates whether the accuracy of the SP 330 is improving (i.e., model drift detection). The accuracy of the SP 330 is expected to improve over time and be maintained at a higher level than at the beginning based on additional SME prediction feedback. If such improvement and sustained improvement in performance does not occur, it may indicate a malfunction of the part of the system 100, 100A, 100B responsible for generating the SP 330 that is not improving and not being maintained at a high accuracy level. If the answer to the inquiry at decision block 514 is no, the method 500 proceeds to block 516 and generates an alert. The method 500 then proceeds to decision block 518. At decision block 518, the method 500 evaluates whether there are additional SPs 330 to be evaluated. If the answer to the inquiry at decision block 518 is yes, the method 500 proceeds to block 520 and ends. If the answer to the inquiry at decision block 518 is no, the method 500 returns to block 504 to access and evaluate the next SP 330 and perform another iteration of the method 500.

[0097] FIG. 6A is a simplified block diagram of a system and / or method 600A according to an embodiment of the present disclosure. FIG. 6A illustrates a non-limiting example of a framework that represents how a calibrated physics-based model 606 can be used to complement a PHM AI engine (part of the digital ecosystem 120, 120A) according to aspects of the present disclosure. Under this proposed framework, historical sensor / actuator data 602 is used to generate statistics that are used to calibrate a physics-based dynamic model 606 of the system or machine under study. This physics-based model 606 is then modified to inject explicit failures in the form of changes in component parameters that have values ​​equivalent to the failure. For example, the pressure drop of a filter can be set to have a value several orders of magnitude higher than its nominal calibration value. Once these conditions are injected into the model, the model will behave as if the filter has reached the end of its life and will generate a sensor / actuator signature that is specific to this failure in most cases. Furthermore, the model is not limited to the set of sensor / actuator parameters available through the collected telemetry. The model can generate additional information, both in the form of additional sensors and signals not available in the telemetry (FIG. 7B) and higher sampling rates of the collected data (FIG. 7C). This expands the capabilities of the AI, as it can now be trained to detect anomalies beyond those experienced by the physical machine, which is the source of the limited historical data set. As the physics model 606 reaches maturity, the model 606 essentially becomes part of the model generated by the PHM AI engine. Ultimately, the physics-based model 606 generates a comprehensive library of sensor signatures associated with specific failure modes and component degradation. This library will drive the evolution of the AI ​​model and the engineering development of future machines.

[0098] One aspect of physics-based models is that they do not need to be fully constructed to begin generating values. For example, a model that includes only the hydraulic domain of the system under study, once calibrated, can begin to generate signatures of sensor faults specific to the hydraulic domain of the machine. Thermal and electromechanical domains can then be added as needed to simulate such physical domains and extract signatures of more complex interactions in the multi-domain system.

[0099] FIG. 6B is a simplified block diagram of a system and / or method 600B according to an embodiment of the present disclosure. The system / method 600B is a modification of the system / method 600A. In some embodiments of the present disclosure, synthetic data may be generated via a physics-based model developed and used via the process 600B shown in FIG. 6B, although additional means and architectures are possible. The physics-based model is used to generate synthetic data (i.e., data that did not originate organically / naturally / from a non-synthetic data generator such as manipulable hardware). The need for synthetic data arises from the fact that the important events, machine conditions, etc. that need to be understood rarely, if ever, occur organically. Thus, in order to understand these rare / infrequent / new events and machine conditions, synthetic data (i.e., sensor fault signatures, 610) must be created that attempt to replicate what the real / non-synthetic data would look like if those events and machine conditions occurred. Primary purposes and uses of synthetic data may include, but are not limited to, training artificial intelligence models to predict / classify / identify / etc. rare / low frequency / new events and machine conditions, performing fault analysis for quality assurance, building probabilistic risk models, performing diagnostic analysis, understanding hardware complexities to enhance and improve existing hardware configurations, etc.

[0100] Under the system / method 600B, an internal domain expert (SME) 603 uses the system schematics and other engineering documents 601 to generate a physics-based model template 604. In some embodiments of the present disclosure, a cognitive algorithmic model can be trained to use the system schematics and other engineering documents 601 to generate the physics-based model template 604. This model template 604 includes all the components of the system being modeled, e.g., pumps, filters, chemical reactors, etc., but lacks the appropriate component parameter values ​​that define the dynamic performance of the real machine. That is, it is not calibrated to operate like the real machine. To obtain a calibrated model, accurate statistics are generated using historical telemetry 602, which are then used by the SME 603 ​​to calibrate the physics-based dynamic model 606 using a feedback-based algorithm. Once calibrated, the model 606 acts as a digital twin of the real hardware (i.e., the system dynamics observed in the model 606 match the system dynamics of the real hardware).

[0101] Once the physics-based dynamic model 606 is created, the model 606 can be used to generate synthetic data sets that show sensor failure signatures and nominal or off-nominal system behavior. This can be done by taking the model 606 and copying it many times to create simulations of different events 607, each modified to account for a given failure mode or behavior. This modification comes in the form of changes to component parameters that have values ​​equivalent to a given failure 608. For example, the pressure drop of a filter can be set to have a value several orders of magnitude higher than its nominal calibration value. Once these conditions are injected into the model, the model will behave as if the filter has reached the end of its life and will generate a sensor / actuator signature that is, in most cases, specific to this failure. After many simulations 607, the generated sensor / actuator signatures can be aggregated into a dictionary of synthetic signals 610 that correspond to specific failures. This dictionary 610 can then be used as a reference point for the prognostic and health monitoring AI engine to identify anomalies, generate prognoses, etc.

[0102] FIG. 6C is a simplified block diagram of a system and / or method 600C according to an embodiment of the present disclosure. More specifically, FIG. 6C illustrates how a physics-based model according to an aspect of the present disclosure can be used to generate synthetic data, more specifically, a signature of a failure mode of a component. Although FIG. 6C presents only an operational flow path for generating a signature of a specific component failure using a physics-based model, this process can be scaled to broader system-level interactions. For example, the signature can be generated at the orbital replacement unit (ORU) or system level. In the non-limiting example shown in FIG. 6C, a failure mode of a specific component is selected for simulation. Once the component is identified, parameters related to the performance and failure of that component are selected. The parameters are modified to represent a failure state, and those parameter "settings" are then injected into the physics-based model. Running the model with the modified parameters generates and stores a signature of the failure mode of interest. If additional signatures are needed to simulate the failure mode, the process is repeated. If not, the result is stored and used to generate a model in the PHM AI engine.

[0103] FIG. 6D is a simplified block diagram of a system and / or method 600D according to an embodiment of the present disclosure. More specifically, FIG. 6D illustrates a non-limiting exemplary use case of a physics-based model for synthetic data generation according to an embodiment of the present disclosure. An example of the creation and use of a physics-based dynamic model 606 can be observed by considering the development of such a model 601b through a process 600D. As shown, an SME 605 wants to create a physics-based model (601b) that represents an actual International Space Station (ISS) Water Treatment Apparatus (WPA) 601a. ​​To do this, the SME 605 uses the WPA system schematic and other engineering documents (e.g., software requirements, specifications, etc.) to generate a basic physics-based model, or referred to as a template 604. This basic physics-based model does not behave like the actual WPA 601a because its modeled components do not have properties equivalent to those of the components in the actual system. To identify how the characteristics need to be modified, the SME considers the observable system dynamics and behavior of the WPA 601a through collected telemetry 602a. The SME then modifies the characteristics (e.g., pressure characteristics [hydraulic diameter], etc.) of the model template 604 until the synthetic data set 602b closely matches the characteristics of the actual hardware telemetry 602a. Once this behavior is achieved, a physics-based model 601b representative of the WPA has been created.

[0104] A physically based model 601b of the WPA can then be used to explore physical phenomena that may otherwise be difficult to observe. The SME can use the model 601b to simulate undesirable running conditions 604 by further modifying the characteristics of 601b. For example, if the SME 605 wants to evaluate what would happen if the filter in the WPA 601a becomes clogged, the SME 605 can modify the hydraulic diameter characteristics of the filter in the model 601b. Once the characteristics are modified to represent the desired running conditions 604, the model is run as a simulation 607. The simulation 607 generates a synthetic data set 602b indicative of what would happen if the filter in 601a becomes clogged. This synthetic data set 602b would then serve as a fault signature indicative of a clogged filter in 601a and would be stored for reference in a sensor fault signature dictionary 610 in the synthetic data repository 603b. The fault signature of a clogged filter can then be used to train the PHM AI engine of the system 100, 100A, 100B to recognize when the filter of the WPA601a will become clogged, or to predict the time until it becomes clogged, etc., by comparing the fault signature with the actual system telemetry 602a of the WPA601a.

[0105] In some embodiments of the disclosure, the AI ​​engine process 700A illustrated in Figure 7A may be implemented in the manner shown, although additional means and architectures are possible. In the AI ​​engine process 700A, a consumer 701 takes in information from either the PHM AI engine metamodel 706 or the flight hardware 707. Upon receiving this information, the consumer 701 ultimately consumes the information, i.e., does not send further information back through 700A, or may interact / pass further data through the system.

[0106] If a consumer desires / is commanded to return information via 700A, there are two possible channels through which the data can be sent. First, the data can be sent back as regular telemetry 702, i.e., the data is intended to be returned to the Telemetry and Data Repository 704 for storage purposes, but no immediate action. Second, the data can be sent back as triggered event telemetry 703, i.e., the data is intended to be returned to the Telemetry and Data Repository 704 with the hope that an immediate action will be taken. If the data streamed through 700A is sent through the trigger event channel 703, the PHM AI Engine's metamodel 706 will use that trigger and the new data to automatically initiate some set of task events (such as updating / retraining an existing model, completely rebuilding an existing model, performing model prediction drift detection, etc.).

[0107] Additionally, the triggered events 703 can also be used to pass information to the physics-based modeling engine 705 via the telemetry and event data repository 704, and the new data can be automatically used to further update / tune / fix / calibrate / validate the modeling engine. Regular telemetry 702 can also be captured by the physics-based modeling engine 705, but since it was not sent as a triggering event 703, such data is manually migrated and processed in the physics-based modeling engine 705. The synthetic data generated by the physics-based modeling engine 705 is then also fed to the PHM AI engine metamodel 706 for use in self-learning.

[0108] As an example of an actual FIG. 7A example, let flight hardware 707 represent the International Space Station ECLSS Water Treatment Assembly (WPA). If consumer 701 is an astronaut who has just changed a filter in the WPA, the consumer can send information about the filter change over channel 703 as a triggered event. The information about the filter change is then stored in telemetry and data event repository 704, which may then automatically initiate a series of events. The new information received about the filter change allows PHM AI engine metamodel 706 to validate its predicted schedule for when the filter will be changed against the actual change date. Additionally, the new information can be fed into the various models associated with the WPA through an automatic retraining event, or an entirely new model and data preprocessing can be created if the metamodel determines it is necessary. Additionally, the triggered event 703 can also stream data through the telemetry and event data repository 704 to the physics-based modeling engine 705, where the conditions surrounding the filter change can be used to automatically validate or calibrate / tune physics-based models of the filter and surrounding systems to ensure that the physics-based models accurately reflect actual flight hardware performance.

[0109] FIG. 7B is a simplified block diagram of a system and / or method 700B according to an embodiment of the present disclosure. In some embodiments of the present disclosure, the simultaneous synthetic sensor (CSS) data generation process 700B depicted in FIG. 7B can be implemented in the manner shown in the figure, although additional means and architectures are possible. CSS is defined as the computation of some measurement in hardware that does not have an explicit sensor (i.e., the sensor is actually embedded in the hardware) and therefore must be estimated. The "sensor" element of CSS refers to the fact that CSS functions exactly as any explicit sensor in that it is a means of capturing real-time measurements. The "synthetic" element of CSS refers to the fact that the data generated by CSS is technically synthetic since there is no explicit sensor for the measurements that CSS computes. The "simultaneous" element of CSS refers to the fact that for each time-stamped point of data captured by an explicit sensor, there is a simultaneous synthetic measurement.

[0110] The intent and purpose of the CSS data generation process is to infer values ​​of key system measurements that are not explicitly measured or collected by the full set of sensors embedded in the hardware. Use cases for CSS data include, but are not limited to: 1) augmenting the telemetry of real / non-synthetic data collected from the hardware to provide additional potentially powerful measurements to be used in all downstream PHM AI Engine models and / or performance analysis; 2) determining which sensors are essential to embed in the hardware such that the number of sensors required is minimized (to reduce cost and engineering complexity) while maximizing the total number of measures / features that can be captured for analytical purposes; and 3) resolving system issues that may be caused by latent measurements not directly observed in the telemetry. Use case 3 is discussed in more detail in FIG. 7C.

[0111] In some embodiments of the present disclosure, the CSS data generation process includes hardware 711 that includes explicit sensors 712 that measure various performance metrics (e.g., temperature, pressure, conductivity, etc.) and send those values ​​back via telemetry to the telemetry and event data repository 713. There are various solutions that may or may not have explicit sensors 712 on the hardware due to cost constraints, engineering complexity, or physical impossibility. As a result, potentially important information may not be present in the telemetry and event data repository 713. For example, a fluid pump may have an explicit sensor 712 that measures the velocity and temperature of the fluid at the outlet, but not a sensor that measures pressure. As a result, the pressure at this point in the system may be considered a potential measurement that may be represented by the CSS. To accurately generate a CSS for pressure values ​​on a system, measurements from the explicit sensors may be passed through a physics-based modeling engine 714 (described in more detail in FIG. 8A ) to generate expected values ​​for pressure (through known physical relationships and some modeling adjustments), thereby generating values ​​for the CSS (i.e., inferred sensors) 715. This estimate is then passed to a synthetic data repository 716 that holds timestamps so that it can be linked 717 to the telemetry and event data repository 713 .

[0112] To prevent confusion as to why the complexity of model 714 is necessary rather than directly utilizing physical equations, physics-based equations are only accurate if all components in the system are perfect. However, due to imperfections and deformations in manufacturing, installation and assembly, and operation, standard physics equations may not be an accurate representation, and therefore direct calculations from explicit sensors to inferential sensors may not be accurate. For example, a pipe originally designed to be perfectly round may be damaged during installation, resulting in the installed pipe actually containing a crimp or recess in its structure. Physical modeling operating without the benefit of the embodiments of the present disclosure defined herein would not be able to generate outputs corresponding to a damaged pipe. The functionality required to address this situation may be provided via model 714, according to embodiments of the present disclosure.

[0113] As an actual example of Figure 7B, let hardware 711 represent the International Space Station ECLSS Water Treatment Assembly (WPA). With reference to Figure 7B, let explicit sensor 712 "a" be the velocity of the liquid at the outlet of the fluid pump, explicit sensor 712 "b" be the temperature of the liquid at the outlet of the fluid pump, absent sensor "c" measure the pressure at the outlet of the fluid pump (if present), and explicit sensor 712 "d" measure the viscosity of the fluid at the outlet of the fluid pump. To estimate the value of the pressure at the outlet, the explicit values ​​of the time-based measurements of sensors 712 "a", "b" and "d" are passed to a physics-based modeling engine 714 that has been trained (specifically calibrated with ISS WPA data) such that model 714 can generate CSS data 715 for "c". When the explicit sensor "a", "b", and "d" values ​​are passed and stored in the telemetry and event data repository 713, the CSS data 715 values ​​derived from the model 714 are time-stamped and sent to the synthetic data repository 716. When the data is extracted for later use to train the prognostic failure model of the fluid pump, the time-stamp linkage 717 between the synthetic data repository 716 and the telemetry and event data repository 713 allows both the explicit sensor data 712 and the CSS data 715 to be merged together to enhance the training data for the model, thus potentially increasing the predictive power of the prognostic model.

[0114] Further exemplary applications of the process illustrated in FIG. 7B are applied in the context of use case 3 (i.e., to solving system problems that may be caused by latent measurements not directly observed in the telemetry) and can be illustrated through FIG. 7C. For example, assume that the telemetry of a space-based product such as the WPA on the ISS is capturing and delivering performance telemetry to mission control and ground SMEs at a sampling frequency of 0.1 Hz (i.e., one data point every 10 seconds). While this sampling rate may be sufficient during periods of nominal performance, some dynamic behaviors may not be observable at this sampling rate. For example, if a pump in the WPA is worn out, the rate at which the pump accelerates to operating speed or decelerates during an outage may indicate the health of the pump. If the pump accelerates to operating speed or decelerates during an outage in a period of less than 10 seconds, the acceleration or deceleration behavior is not observable in the data. By running the captured telemetry through a physics-based modeling engine, a high-resolution synthetically supplemented data set can be generated (i.e., synthetic data points can fill in gaps in the data). This new data set can then be used to study the acceleration or deceleration behavior of the pump, so that the health of the pump can be investigated and inferred for prognostic and problem solving purposes.

[0115] FIG. 8A is a simplified block diagram of a system and / or method 800A according to an embodiment of the present disclosure. In some embodiments of the present disclosure, the synthetic data generation process 800A illustrated in FIG. 8A can be implemented in the manner illustrated in the figure, although additional means and architectures are possible. The intent and purpose of the synthetic data generation process is to create synthetic data (i.e., data that does not arise organically / naturally / from a non-synthetic data generator). The need for synthetic data arises from the fact that important events, machine conditions, etc. that need to be understood rarely, if ever, occur organically. Therefore, in order to understand these rare / infrequent / novel events and machine conditions, synthetic data must be created that attempts to replicate what the actual / non-synthetic data would look like if those events and machine conditions occurred. Primary purposes and uses of synthetic data can include, but are not limited to, training artificial intelligence models to predict / classify / identify / etc. rare / infrequent / novel events and machine conditions, performing fault analysis for quality assurance, building probabilistic risk models, performing diagnostic analysis, understanding hardware complexities to enhance and improve existing hardware configurations, etc.

[0116] In some embodiments of the present disclosure, the synthetic data generation process consists of source data 801 that gives rise to real / non-synthetic data. The source data 801 can include, but is not limited to, hardware, internal domain experts, users or hardware of the present disclosure, etc. The real / non-synthetic data is then captured through various means into the telemetry and event data repository 802, which is part of the data repository of the system 100. This real / non-synthetic data can then be utilized in two separate means to create synthetic data. First, non-physics-based models 808 (e.g., generative adversarial networks, autoencoders, etc.), which are models that do not have explicit programming related to the physics of the hardware, can be trained on this data. Once these models are trained and can accurately emulate the nominal performance of the organic data, rare / infrequent / new events and machine states can be faked using input parameter values ​​that match the desired events / machine states. The output of the models 808 will consist of synthetically generated data that theoretically reflects the performance behavior of the underlying hardware. The output of 808 is then captured as part of a synthetic data repository 809 that is part of the data repository of the system 100 and is later used in the PHM AI engine process described below in FIG. 8B.

[0117] The second means by which synthetic data may be generated is by using a physics-based modeling engine 807. The physics-based modeling engine 807 consists of two components. The first component is an auto-calibrating physics-based model 805. The model 805 itself consists of two parts: a physics-based model 805a (described previously herein) and a calibration model 805b. The physics-based model 805a is a dynamic model that leverages the known physics and assembly of the hardware to mathematically represent the performance of the hardware. The physics-based model 805a is a theoretical model that ultimately aims to digitally reconstruct the performance of the hardware based on its physics, effectively acting as a hardware digital twin, as previously described herein. However, this model may not fully reflect the actual performance of the hardware in service due to slight variations or modifications in the hardware itself, unknown installation and assembly differences, etc. As a result, the calibration model 805b is used in conjunction with the physics system 805 to adjust the physics-based model to accurately reflect the true hardware performance. This calibration model 805b can be constructed in a variety of ways, such as traditional calibration methods from the signal processing field, gradient descent optimization, or reinforcement learning artificial intelligence designed to optimize the resulting distributions, correlations, autocorrelations, etc.

[0118] A secondary component of the physics-based modeling engine 807 is the error adjustment model 806. Even after being optimized and calibrated, the data generated by the auto-calibrated physics-based model 805 may not perfectly emulate the true machine. Thus, a second set of models (statistical or artificial intelligence) may be utilized to correct the output of the model 805, but may not be utilized. The error adjustment model 806 can learn to predict the error between the synthetic data output of the model 805 and the real / non-synthetic data by a process of supervised machine learning or other predictive models. Thus, the synthetic data created by the model 805 can be passed through the model 806 to generate a final state of synthetic data that is ultimately sent to a synthetic data repository 809, which is part of the data repository of the system 100. Once all the elements within the physics-based modeling engine 807 are trained and calibrated, it can initiate the desired rare / low frequency / new events and machine states and ultimately generate synthetic data that reflects what the data would look like under the conditions of the input states / events / conditions.

[0119] An additional feature of the synthetic data generation process 800A is its automatic self-calibrating nature. The need for self-calibration is a) to eliminate the need for interaction with SMEs, which is often delayed or time-consuming; and b) to ensure that the data supporting the PHM AI engine is as accurate as possible for the current hardware performance (e.g., in the case of life support systems, there is a risk that failure prediction models trained on old data that no longer reflects the current hardware performance will produce extreme results). There are two types of calibration events. First, scheduled calibration and training 803 may occur. Such calibration occurs on a set schedule, incorporating the latest accumulated actual / non-synthetic data, and then a) verifies that the physics-based modeling engine 807 is within tolerance of the observed actual / non-synthetic data, and b) retrains and updates the model 808. Second, event-triggered calibration and training 804 may occur. Certain events, such as the occurrence of a known significant event, may automatically initiate a calibration validation and retraining cycle unscheduled. Again, once a calibration cycle is initiated, a) the physics-based modeling engine 807 is validated against the observed data to ensure it is within tolerance of the observed actual / non-synthetic data, and b) the model 808 is retrained. If drift occurs between the physics-based modeling engine 807 and the observed actual / non-synthetic data, both models 805b and 806 are automatically retrained and reoptimized to bring the physics-based modeling engine 807 into line with the performance of the actual hardware.

[0120] As an example of an actual FIG. 8A example, let source data 801 represent the International Space Station ECLSS Water Treatment Assembly (WPA). Let models 807 and 808 already exist and have been optimized. If, after models 807 and 808 have been trained, NASA decides to increase the constant flow rate of water through the WPA, it is expected that the flight hardware will begin to deviate from its modeled performance because the increased flow rate may denature and distort the physical components inside the WPA in ways that models 807 and 808 cannot capture. As a result, after a scheduled 803 or triggered 804 calibration and training cycle, the latest actual WPA data is extracted from the telemetry and event data repository 802. The non-physics-based model 808 can then be retrained with this new data to learn the latest deviations in the WPA hardware and emulate its performance. Additionally, the validation cycle of the physics-based modeling engine 807 may conclude that the physics-based modeling engine 807 is no longer within tolerance of the observed / non-synthetic data, causing the physics-based modeling engine 807 to automatically retrain the calibration model 805b and the error adjustment model 806 to bring it back into alignment with the observed / non-synthetic data so that it remains a viable digital twin. Once 807 is brought back into alignment with the actual WPA data and the model 808 is retrained, a number of rare / infrequent / new events and machine conditions of interest, such as failure of critical pump "X", can be simulated through models 807 and 808 to understand what the data would look like under new operating conditions of higher constant flow. This synthetic data can then be fed into the PHM AI engine to (re)train the model in the engine, which can predict in advance if critical pump "X" will fail, providing advanced warning to astronauts so they can take action to protect and preserve their lives.

[0121] FIG. 8B is a simplified block diagram of a system and / or method 800B according to an embodiment of the present disclosure. In some embodiments of the present disclosure, the AI ​​engine process illustrated in FIG. 8B may be implemented in the manner illustrated in the figure, although additional means and architectures are possible. In some embodiments of the art, the PHM AI engine is defined as a set of models created, modified, optimized, etc. by a larger model (metamodel). The PHM AI engine is a multi-layer system, and the metamodel creates and interacts with various AIML and other statistical models that can be deployed for use in the digital ecosystem. The metamodel itself is not directly used or consumed by the digital ecosystem, but creates / trains / optimizes / etc. models that can and are consumed by the digital ecosystem. In the AI ​​engine process 800B, the consumer 811 takes in information from either the PHM AI engine metamodel 816 or flight hardware 817. Upon receiving this information, the consumer 811 ultimately consumes the information, i.e., does not send further information back through 800B, or may interact / pass further data through the system.

[0122] If the consumer desires and / or is commanded to return information via 800B, there are two possible channels through which the data can be sent. First, the data can be sent back as regular telemetry 812, i.e., the data is intended to be returned to the Telemetry and Data Repository 814 for storage purposes, but no immediate action. Second, the data can be sent back as triggered event telemetry 813 (i.e., the data is intended to be returned to the Telemetry and Data Repository 814 with the hope that an immediate action will be taken). If the data streamed through 800B is sent through the trigger event channel 813, the PHM AI Engine's metamodel 816 will use the trigger and the new data to automatically initiate some task event (such as updating / retraining an existing model, completely rebuilding an existing model, performing model prediction drift detection, etc.).

[0123] Additionally, triggered events 813 can also be used to pass information to the physics-based modeling engine 815 via telemetry and event data repository 814, and the new data can be automatically used to further update / tune / fix / calibrate / validate the modeling engine. Regular telemetry 812 can also be ingested into the physics-based modeling engine 815, but since it was not sent as a triggering event 813, such data is manually migrated and processed in the physics-based modeling engine 815 via scheduled events. Synthetic data generated by the physics-based modeling engine 815 is also fed to the PHM AI engine metamodel 816 (through the intermediation of synthetic data repository 815a) and used for self-learning.

[0124] As an example of an actual Figure 8B, let flight hardware 817 represent the International Space Station's ECLSS Water Treatment Assembly (WPA). If consumer 811 is an astronaut who has just changed a filter in the WPA, the consumer can transmit information about the filter change over channel 813 as a triggered event. The information about the filter change is then stored in telemetry and data event repository 814, after which a series of events may automatically begin.

[0125] New information received about a filter change can cause the PHM AI engine metamodel 816 to validate its predicted schedule for when that filter will be changed against the actual change date. Additionally, the new information can be fed into the various models associated with the WPA through an automatic retraining event, or an entirely new model and pre-processing of the data can be created if the metamodel determines it is necessary. Additionally, the triggered event 813 can also flow data through the telemetry and event data repository 814 to the physics-based modeling engine 815, where the conditions surrounding the filter change can be used to automatically validate or calibrate / tune the physics-based models of the filter and surrounding systems to ensure that the physics-based models accurately reflect the actual flight hardware performance.

[0126] Thus, from the foregoing detailed description, it can be seen that embodiments of the present disclosure provide technical effects. For example, the functionality implemented by the system 100, 100A enables current staff and SMEs to support an increasing number of products and platforms. The system 100, 100A uses the SME model and / or SME digital twin to provide new workers with fingertip access to the digitized contextual knowledge of the SMEs, minimizing training and learning curves (time to become an expert), enabling them to provide reliable product support. The knowledge of the SMEs is retained in the algorithms and contextual information within the intelligent system 100, 100A, meaning that the experience and knowledge of the SMEs is not wasted. The system 100, 100A enables simplified controller solutions, allows for control and monitoring of commercially available ECLSS products from the ground, lowers controller costs, and enables price-to-win targeting.

[0127] The architecture of the system 100, 100A can equip existing products with intelligent product features such as predictive and health management capabilities without retrofitting. The ability of the system 100, 100A to evaluate real-time data helps identify problems faster. The model enhancement capabilities of the system 100, 100A (e.g., via QA systems, SME models, SME digital twins, etc.) provide enhanced model accuracy, foresight, and fine-grained oversight.

[0128] Additional details of machine learning techniques that can be used in the disclosed aspects of the present disclosure are provided herein. Various types of computer control functions of the processors described herein can be implemented using machine learning, deep learning, and / or natural language processing techniques. In general, deep learning techniques can be performed on so-called "neural networks," which can be implemented as programmable computers configured to execute a set of dedicated algorithms. Neural networks are data agnostic and applicable to a variety of fields, including neurophysiology, cognitive science / psychology, physics (statistical mechanics), control theory, computer science, artificial intelligence, statistics / mathematics, pattern recognition, computer vision, parallel processing, and hardware (e.g., digital / analog / VLSI / optics).

[0129] The basic function of neural networks and their machine learning algorithms is to recognize patterns by interpreting structured data through some kind of machine perception. Unstructured real-world data in its native format (e.g., image, audio, text, or time series data) is converted into a structured numerical format (e.g., vectors with magnitude and direction) that can be understood and manipulated by a computer. Machine learning algorithms perform multiple iterations of learning-based analysis on real-world data vectors until patterns (or relationships) contained in the real-world data vectors are revealed and learned. The learned patterns / relationships serve as predictive models that can be used to perform a variety of tasks, including, for example, classification (or labeling) of real-world data and clustering of real-world data. Classification tasks often rely on the use of labeled datasets to train neural networks (i.e., models) to recognize correlations between labels and data. This is known as supervised learning. Examples of classification tasks include detecting people / faces in an image, recognizing facial expressions in an image (e.g., angry, happy, etc.), identifying objects in an image (e.g., stop signs, pedestrians, lane markers, etc.), recognizing gestures in a video, detecting speech, detecting voices in audio, identifying specific speakers, converting speech to text, etc. Clustering tasks identify similarities between objects, grouping them according to their common features, and using these features to distinguish them from other groups of objects. These groups are known as "clusters".

[0130] One example of a machine learning technique that can be used to implement aspects of the present disclosure is described with reference to Figures 9 and 10. A machine learning model configured and arranged in accordance with an embodiment of the present disclosure is described with reference to Figure 9. A detailed description of an example computing system and network architecture capable of implementing one or more of the embodiments of the disclosure described herein is provided with reference to Figure 11.

[0131] FIG. 9 illustrates a block diagram showing a machine learning system 900 that can implement various aspects of the disclosure described herein (e.g., regression, classification, anomaly detection, clustering, autonomous decision making, etc.). More specifically, the functionality of the system 900 is used in embodiments of the disclosure to generate various models and sub-models that can be used to implement computer functions in embodiments of the disclosure. The system 900 includes multiple data sources 902 that communicate with an AI algorithm 910 via a network 904. In some aspects of the disclosure, the data sources 902 can bypass the network 904 and feed directly to the AI ​​algorithm 910. The data sources 902 provide inputs of data / information used by the AI ​​algorithm 910 according to embodiments of the disclosure. The data sources 902 also provide inputs of data / information that can be used by the AI ​​algorithm 910 to train and / or update the model(s) 916 created by the AI ​​algorithm 910. The data sources 902 can be implemented as a variety of data sources, including, but not limited to, sensors configured to collect real-time data, data repositories (including training data repositories), and outputs from other AI algorithms. The network 904 can be any type of communication network, including, but not limited to, a local network, a wide area network, a private network, the Internet, etc.

[0132] The AI ​​algorithms 910 may be implemented as algorithms executed by a programmable computer, such as the processing system 1100 (shown in FIG. 11). As shown in FIG. 9, the AI ​​algorithms 910 include machine learning (ML) algorithms (including classical statistical learning models, deep learning models, reinforcement learning models, and any combination thereof) 912, natural language processing (NLP) algorithms 914, and a set of other types of model(s) 916 that are relational (or predictive) algorithms generated (or learned) by the ML algorithms 912. The algorithms 912, 914, 916 of the AI ​​algorithms 910 are shown separately for ease of illustration and explanation. In embodiments of the present disclosure, the functions performed by the various algorithms 912, 914, 916 of the AI ​​algorithms 910 may be distributed differently than shown. For example, if the AI ​​algorithms 910 are configured to perform an overall task having subtasks, the set of ML algorithms 912 may be segmented such that some of the ML algorithms 912 perform each subtask and some of the ML algorithms 912 perform the entire task. Further, in some embodiments of the present disclosure, the NLP algorithms 914 may be integrated within the ML algorithms 912.

[0133] The NLP algorithms 914 include speech recognition functionality that enables the AI ​​algorithms 910, and more specifically the ML algorithms 912, to receive natural language data (text and speech) and apply elements of language processing, information retrieval, and machine learning to derive meaning from the natural language input and potentially take action based on the derived meaning. The NLP algorithms 914 used in accordance with aspects of the present disclosure can also include speech synthesis functionality that enables the AI ​​algorithms 910 to convert result(s) 920 into natural language (text and audio) to communicate aspects of the result(s) 920 as natural language communication.

[0134] The NLP and ML algorithms 914, 912 receive and evaluate input data (i.e., training data and data under analysis) from the data sources 902. The ML algorithm 912 includes the functionality necessary to interpret and utilize the format of the input data. For example, if the data sources 902 include image data, the ML algorithm 912 may include visual recognition software configured to interpret the image data. The ML algorithm 912 applies machine learning techniques to the received training data (e.g., data received from one or more of the data sources 902) to create / train / update, over time, one or more models 916 that model the overall task and subtasks that the AI ​​algorithm 910 is designed to complete.

[0135] 9 and 10 together, FIG. 10 illustrates an example of a training phase 1000 performed by the ML algorithm 912 to generate the model 916 described above. In the training phase 1000, the AI ​​algorithm 910 extracts features from the training data and converts the features into a vector representation that can be recognized and analyzed by the ML algorithm 912. The feature vectors are analyzed by the ML algorithm 912 to reveal relationships between and among the training data. Examples of suitable implementations of the ML algorithm 912 include, but are not limited to, linear / polynomial / ridge / lasso / elastic net regression, logit regression, k-nearest neighbors, decision trees, random forests, gradient boosted forests, naive Bayes classifiers, support vector machines / regressors, neural networks (traditional artificial neural networks, recurrent neural networks, convolutional neural networks, generative adversarial networks, etc.), Q-learning, policy gradient models, k-means, DBSCAN clustering, hidden Markov models, etc. The learning or training performed by the ML algorithm 912 may be generative, supervised, unsupervised, hybrid including aspects of supervised learning and unsupervised learning (semi-supervised), or reinforcement models.

[0136] Once the Model 916 is sufficiently trained by the ML algorithm 912, it accesses the data source 902 that generates "real world" data, similar to the data source 902 that generates synthetic data, and applies the data to the Model 916 to generate a usable version of the result 920. In some embodiments of the present disclosure, the result 920 can be fed back to the AI algorithm 910 and used by the ML algorithm 912 as additional training data to update and / or refine the Model 916.

[0137] In aspects of the present disclosure, the ML algorithm 912 and the Model 916 can be configured to apply a confidence level (CL) to various ones of their results / judgments (including the result 920) to improve the overall accuracy of a particular result / judgment. When the ML algorithm 912 and / or the Model 916 make a judgment or generate a result where the value of the CL is below a predetermined threshold (TH) (i.e., CL < TH), that result / judgment can be classified as having a "confidence level" low enough to justify the conclusion that the judgment / result is not valid, and this conclusion can be used to determine when, how, and / or in what downstream processing the judgment / result is handled. When CL > TH, the decision / result can be considered valid, and this conclusion can be used to determine when, how, and / or whether the decision / result is processed in downstream processing. A number of different predetermined TH levels can be provided. Decisions / results with CL > TH can be ranked from the highest CL > TH to the lowest CL > TH to prioritize when, how, and / or whether to process the decision / result in downstream processing.

[0138] In aspects of the present disclosure, the AI algorithm 910 can be configured to apply a confidence level (CL) to the result 920. When the AI algorithm 910 determines that the CL of the result 920 is below a predetermined threshold (TH), i.e., CL < TH, the result 920 can be classified as being low enough to justify a "not reliable" classification of the result 920. If CL > TH, the result 920 can be classified as being high enough to justify a determination that the result 920 is valid. A number of different predetermined TH levels can be provided so that results 920 with CL > TH can be ranked from the highest CL > TH to the lowest CL > TH.

[0139] The functions performed by the AI algorithm 910, and more specifically, the functions performed by a neural network (one possible representation of which is the ML algorithm 912), can be organized as a weighted directed graph, where the nodes are artificial neurons (e.g., modeled after the neurons of the human brain) and the weighted directed edges connect the nodes. The directed graph of the AI algorithm 910 can be organized such that certain nodes form the input layer nodes, certain nodes form the hidden layer nodes, and certain nodes form the output layer nodes. The input layer nodes are coupled to the hidden layer nodes, and the hidden layer nodes are coupled to the output layer nodes. Each node is connected to all the nodes in the adjacent layer by connection paths, which can be depicted as directed arrows each having a connection strength. Multiple input layers, multiple hidden layers, and multiple output layers can be provided. When multiple hidden layers are provided, the AI algorithm 910 can perform deep learning to execute the assigned task(s) of the AI algorithm 910.

[0140] Similar to the functions of the human brain, each input layer node receives an input without connection strength adjustment and without node summation. Each of the hidden layer nodes receives its input from all the input layer nodes according to the connection strength associated with the relevant connection path. Similar connection strength multiplication and node summation are also performed for the hidden layer nodes and the output layer nodes.

[0141] One common form of neural network processes data records (e.g., output from data source 902) one at a time, and it "learns" by comparing an initially arbitrary classification of the record to the known actual classification of the record. Using a training method known as "backpropagation of error" (i.e., "backpropagation of error"), errors from the initial classification of the first record are fed back into the weighted directed graph of the AI ​​algorithm 910 and used a second time to modify the weighted connections of the weighted directed graph, and this feedback process continues for many iterations. During the training phase of the weighted directed graph of the AI ​​algorithm 910 for supervised learning problems, the correct classification of each record is known, so that the output nodes can be assigned "correct" values. For example, the node value of the node corresponding to the correct class is "1" (or 0.9), and the node value of the other classes is "0" (or 0.1). It is therefore possible to compare the calculated values ​​of the weighted directed graph of the output nodes to these "correct" values ​​and calculate an error term (i.e., the "delta" rule) for each node. These error terms are then used to adjust the weights in the hidden layer so that at the next iteration, the output values ​​are closer to the "correct" values.

[0142] 11 illustrates a high-level block diagram of a computer system 1100 that can be used to perform one or more computer processing operations according to aspects of the disclosure. Although one exemplary computer system 1100 is illustrated, the computer system 1100 includes a communication path 1125 that connects the computer system 1100 to additional systems (not shown) and may include one or more wide area networks (WANs) and / or local area networks (LANs), such as the Internet, intranet(s), and / or wireless communication network(s). The computer system 1100 and the additional systems communicate over the communication path 1125, for example, to communicate data between them.

[0143] The computer system 1100 includes one or more processors, such as processor 1102. The processor 1102 is connected to a communications infrastructure 1104 (e.g., a communications bus, a crossover bar, or a network). The computer system 1100 may include a display interface 1106 that transfers graphics, text, and other data from the communications infrastructure 1104 (or from a frame buffer, not shown) for display on a display device 1108. The computer system 1100 also includes a main memory 1110, preferably random access memory (RAM), and may also include a secondary memory 1112. The secondary memory 1112 may include a hard disk drive 1114 and / or a removable storage drive 1116, representing, for example, a floppy disk drive, a magnetic tape drive, or an optical disk drive. The removable storage drive 1116 reads from and / or writes to a removable storage unit 1118 in a manner well known to those skilled in the art. The removable storage unit 1118 represents, for example, a floppy disk, compact disk, magnetic tape, or optical disk, flash drive, solid-state memory, etc. that can be read by and written to by the removable storage drive 1116. As will be appreciated, the removable storage unit 1118 includes computer readable media having computer software and / or data stored thereon.

[0144] In alternative embodiments of the present disclosure, the secondary memory 1112 may include other similar means for allowing computer programs or other instructions to be loaded onto the computer system. Such means may include, for example, a removable storage unit 1120 and an interface 1122. Examples of such means may include program packages and package interfaces (such as those found in video game consoles), removable memory chips (such as EPROMs, or PROMs) and associated sockets, and other removable storage units 1120 and interfaces 1122 that allow software and data to be transferred from the removable storage unit 1120 to the computer system 1100.

[0145] Computer system 1100 may also include a communications interface 1124. Communications interface 1124 allows software and data to be transferred between the computer system and external devices. Examples of communications interface 1124 may include a modem, a network interface (such as an Ethernet card), a COM port, a PCM-CIA slot and card, and the like. The software and data transferred via communications interface 1124 is in the form of signals, which may be, for example, electrical, electromagnetic, optical, or other signals that may be received by communications interface 1124. These signals are provided to communications interface 1124 via communications path (i.e., channel) 1125. Communications path 1125 carries signals and may be implemented using wire or cable, fiber optics, a phone line, a cellular phone link, an RF link, and / or other communications channels.

[0146] Cloud computing system 50 is in wired or wireless electronic communication with computer system 1100. Cloud computing system 50 may supplement, support, or replace some or all of the functionality (in any combination) of computing system 1100. Additionally, some or all of the functionality of computer system 1100 may be implemented as a node of cloud computing system 50.

[0147] Many of the functional units of the systems described herein are labeled as modules. The embodiments of the present disclosure apply to a wide variety of module implementations. For example, a module may be implemented as a hardware circuit including custom VLSI circuits or gate arrays, off-the-shelf semiconductors such as logic chips, transistors, or other discrete components. A module may also be implemented in programmable hardware devices such as field programmable gate arrays, programmable array logic, programmable logic devices, etc. A module may also be implemented in software for execution by various types of processors. An identified module of executable code may include one or more physical or logical blocks of computer instructions, which may be organized, for example, as an object, procedure, or function. Nevertheless, executable files for an identified module need not be physically located together, but may include disparate instructions stored in different locations that, when logically combined, function as a module and achieve the stated purpose of the module.

[0148] The various components / modules / models of the systems shown herein are depicted separately for ease of illustration and explanation. In embodiments of the present disclosure, the functions performed by the various components / modules / models may be distributed in a manner different from that shown, unless otherwise stated, without departing from the scope of the various embodiments of the disclosure described herein.

[0149] Aspects of the present disclosure may be embodied as systems, methods, and / or computer program products at any possible level of integration technical detail. The computer program product may include a computer-readable storage medium(s) having computer-readable program instructions thereon for causing a processor to execute aspects of the present disclosure.

[0150] A computer-readable storage medium may be a tangible device capable of holding and storing instructions for use by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. A non-exhaustive list of more specific examples of computer-readable storage media includes portable computer diskettes, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), static random access memories (SRAM), compact disk read-only memories (CD-ROMs), digital versatile disks (DVDs), memory sticks, floppy disks, mechanically encoded devices such as punch cards, or raised structures in grooves having instructions recorded thereon, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium is not to be construed as being a transitory signal itself, such as an electric wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a wave guide or other transmission medium (e.g., light pulses passing through a fiber optic cable), or an electrical signal transmitted over an electrical wire.

[0151] The computer readable program instructions described herein can be downloaded from the computer readable storage medium to the respective computing / processing device or to an external computer or storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical transmission fiber, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface of each computing / processing device receives the computer readable program instructions from the network and transfers the computer readable program instructions for storage in the computer readable storage medium within the respective computing / processing device.

[0152] The terms "about," "substantially," and their equivalents are intended to include the degree of error associated with measurement of a particular quantity based on equipment available at the time of filing this application.

[0153] The terms used herein are for the purpose of describing particular embodiments only and are not intended to be limiting of the disclosure. As used herein, the singular forms "a," "an," and "the" are intended to include the plural forms unless the context clearly dictates otherwise. It will be further understood that the terms "comprises" and / or "comprising," as used herein, specify the presence of stated features, components, steps, operations, elements, and / or ingredients, but do not exclude the presence or addition of one or more other features, components, steps, operations, elements, ingredients, and / or groups thereof.

[0154] Although the present disclosure has been described with reference to one or more exemplary embodiments, it will be understood by those skilled in the art that various changes may be made and equivalents may be substituted for the elements thereof without departing from the scope of the present disclosure. In addition, many modifications may be made to adapt a particular situation or material to the teachings of the present disclosure without departing from the essential scope of the present disclosure. Thus, the present disclosure is not limited to the particular embodiment disclosed as the best mode contemplated for carrying out the disclosure, but the present disclosure is intended to include all embodiments falling within the scope of the appended claims.

Claims

1. 1. A system operable to electronically monitor a mission, comprising: A processor system; a digital ecosystem (DE) in electronic communication with the processor system, the DE being associated with the mission and located at a remote station; the DE is operable to generate performance data associated with the deployed product in response to sensed data associated with the deployed product locally deployed with the mission; the processor system includes a local processor system resource that is locally located with the mission; the processor system further includes a remote processor resource at the remote station located remotely from the mission; The processor system is operable to perform processor system operations, the processor system operations comprising: executing a first cognitive algorithm operable to generate an initial cognitive output associated with an initial cognitive output action in response to a portion of the performance data; displaying the initial cognitive output to a user prior to commencing an initial cognitive output action associated with the initial cognitive output; in response to receiving user feedback regarding the initial cognitive output, performing a post-user feedback operation based at least in part on the user feedback; The system comprising:

2. The system of claim 1 , wherein the first cognitive algorithm comprises one or more of an anomaly detection cognitive algorithm, a prognosis cognitive algorithm, and a performance evaluation cognitive algorithm.

3. The system of claim 1 , wherein the user feedback comprises a user's agreement with the initial cognitive output.

4. The system of claim 3 , wherein the post-user feedback operation includes initiating the initial cognitive output action.

5. The system of claim 1 , wherein the user feedback comprises a modification of the initial cognitive output.

6. The system of claim 1 , wherein the user feedback comprises a modification of the initial cognitive output action.

7. The user feedback, Modifying the initial cognitive output; and and modifying the initial cognitive output action.

8. said post-user-feedback operations including training a user model using said user feedback, said user model including an electronic model of said user; the user model is trained to perform a task including generating user-model generated feedback including a prediction of feedback generated by the user in response to instances of the initial cognitive output; The system of claim 1 , wherein the user model includes a digital twin of the user.

9. The system of claim 1 , wherein the first cognitive algorithm comprises one or more of an anomaly detection algorithm, a prognostic algorithm, and a performance evaluation algorithm.

10. and wherein the processor system operations further include executing a second cognitive algorithm to evaluate sufficiency of the user feedback; The sufficiency of the user feedback comprises: Whether the user feedback includes a rationale for the user feedback; and The system of claim 1 , further comprising whether the rationale meets one or more predetermined criteria.

11. The system of claim 10 , wherein the second cognitive algorithm issues a query to the user in response to the second cognitive algorithm determining that the user feedback is insufficient.

12. The inquiry request from the user is the rationale for the user feedback; and and supplementing the rationale of the user feedback.

13. the second cognitive algorithm includes a Q&A algorithm having a natural language processing function; The system of claim 10 , wherein the user feedback includes a natural language rationale for the user feedback.

14. The system of claim 1 , wherein the processor system operations further comprise: generating a knowledge corpus for the user by storing results of operations after the user feedback in a data repository.

15. 1. A system operable to electronically monitor a Near Earth Space Exploration (NESE) mission, comprising: A processor system; a digital ecosystem (DE) in electronic communication with the processor system, the DE being associated with the NESE mission and located at a remote station; The DE is operable to generate performance data associated with the deployed product in response to sensed data associated with the deployed product locally deployed with the NESE mission; the processor system includes local processor system resources located locally with the NESE mission; the processor system further includes a remote processor resource at the remote station located on Earth; The processor system is operable to perform processor system operations, the processor system operations comprising: executing a first cognitive algorithm operable to generate an initial cognitive output associated with an initial cognitive output action in response to a portion of the performance data; displaying the initial cognitive output to a user at the remote station prior to commencing the initial cognitive output action associated with the initial cognitive output; in response to receiving user feedback regarding the initial cognitive output, performing a post-user feedback operation based at least in part on the user feedback; The system comprising:

16. The deployment product includes a first deployment product; the sensed data includes first sensed data associated with the first deployment product; 16. The system of claim 15, wherein operations of the processor system further include executing a physics-based perception algorithm operable to simulate performance of the first deployed product in response to the first sensory data to generate enhanced simulated performance data for the first deployed product.

17. 17. The system of claim 16, wherein executing the physics-based perception algorithm comprises retraining the physics-based perception algorithm based at least in part on the first sensory data associated with the first deployment product.

18. The system of claim 16 , wherein the physics-based perception algorithm comprises a generative perception algorithm.

19. 20. The system of claim 18, wherein the post-user feedback operation includes using the user feedback to train a user model that includes an electronic model of the user.

20. the user model is trained to perform a task including generating user-model generated feedback including a prediction of feedback generated by the user in response to instances of the initial cognitive output; 20. The system of claim 19, wherein the user model includes a digital twin of the user.

Images