Interconnected digital engineering and certification ecosystem

Abstract

We provide interconnected digital engineering, certification ecosystems, and methods. [Solution] A method implemented by a computer in an authentication ecosystem, comprising: receiving design and engineering data (D / E data) corresponding to a prototype representation of a product from a user device; transmitting input derived from the D / E data to a digital engineering tool; receiving engineering-related data from the digital engineering tool; receiving data corresponding to a common validation and verification (V&V) product; identifying product requirements; determining whether the requirements have been met; presenting engineering-related data output and information corresponding to data corresponding to a common V&V product; receiving instructions from a user device; and performing operations on the D / E data.

Description

Technical Field

[0001] Related Applications This application claims priority to U.S. Provisional Patent Application No. 63 / 419,051, filed October 25, 2022, entitled INTERCONNECTED DIGITAL ENGINEERING AND CERTIFICATION ECOSYSTEM.

[0002] This disclosure relates to tools for the certification of digital engineering and digitally engineered products (including modeling and simulation applications).

Background Art

[0003] Digital engineering tools, including modeling and simulation tools that accurately virtualize physical systems or processes for real-world decision-making, enable the agile development of components and / or systems. The certification of these components and / or systems still largely takes place in the physical world using the physical manifestation of digitally engineered components and / or systems (which may sometimes be generally referred to herein as "products").

Summary of the Invention

Means for Solving the Problems

[0004] This specification describes an interconnected digital engineering and certification ecosystem that has several advantages over existing technologies for designing, engineering, testing, and certifying products.

[0005] In recent years, digital engineering tools such as modeling and simulation (M&S) tools, computer-aided design (CAD) tools, model-based systems engineering (MBSE) tools, augmented reality (AR) tools, product lifecycle management (PLM) tools, simulation engines, requirements models, electronics models, test planning models, cost models, scheduling models, software modeling, supply chain models, manufacturing models, cybersecurity models, multi-attribute trade-space tools, and mission effects models have enhanced the agility of hardware development and manufacturing by virtualizing physical systems and / or processes for real-world decision-making. However, given the current state of these digital engineering tools, several challenges remain.

[0006] Firstly, the existence of numerous diverse digital engineering tools (often designed by different parties) presents interoperability challenges and can lead to vendor lock-in issues. In particular, directly integrating individual digital engineering tools with each other is costly in terms of both time and money, and the number of interfaces between digital engineering tools increases in proportion to the square of the number of different digital engineering tools (i.e., N). 2(Computational complexity of ). The numerous and diverse digital engineering tools that exist can also present challenges in implementing scalable applications, automation, machine learning, and / or artificial intelligence across these digital engineering tools. Better interoperability between digital engineering tools can be crucial in product development, testing, and certification processes that may involve several different digital engineering tools used in parallel or sequentially. Thus, seamless interoperability between digital engineering tools may be desirable to implement such processes by enabling the development of “digital threads” or pipelines that connect the inputs and outputs of multiple digital engineering tools for a particular task.

[0007] Secondly, due to the highly technical nature of many digital engineering tools, effectively operating such tools often requires a highly specialized skill set, which limits the number of individuals qualified to use these tools. Furthermore, an individual skilled in using one digital engineering tool (e.g., a CAD tool manufactured by a first software company) may not be qualified to use a different type of digital engineering tool (e.g., an MBSE tool), or even a similar digital engineering tool manufactured by a different company (e.g., a CAD tool manufactured by a second software company). This applies not only to the use of the tools through their custom graphical user interfaces, but also to their use through their tool-specific or vendor-specific APIs, which can also require a highly specialized skill set.

[0008] Thirdly, products and solutions designed using a particular digital engineering tool may not be shareable across digital engineering tools (for example, due to a lack of interoperability). In some cases, previously designed products and solutions may not be shareable or searchable by others using the same digital engineering tools to solve similar problems. For example, a repository of previously designed products and solutions may not exist for sharing information about such products and solutions among individuals within the same team, company, or technical field. Furthermore, even if such a repository of previously designed products and solutions exists, it is unlikely to contain information about how and why those products and solutions were arrived at, or to contain a simple way to reuse previous engineering work from models, which could potentially be used to limit redundant effort and / or provide useful insights to individuals working on similar, but slightly different, products or problems. This can result in many engineering problems requiring development from scratch rather than being built upon the results of past efforts.

[0009] Fourth, products and solutions designed using digital engineering often require the use of many different tools that not everyone knows how to use. For example, a model may be built using a specific MBSE tool, and someone who needs access to the model (or data generated from the model) may not know how to use that tool. This problem, combined with the fact that many complex systems use many different types of tools, means that in order to understand such a system, an individual may have to know how to use many different tools, which may be extremely rare. This problem is further exacerbated by the fact that the person reviewing the information for product certification may not be familiar with some or all of the digital engineering tools and may try to review all the data in legacy formats (such as PDF reports). This poor usability between different modeling tools can cause significant delays and cost increases when developing new products, especially when different people or organizations have different technical skill sets, as models cannot be easily shared between those people or organizations.

[0010] For the reasons stated above, most digital engineering tools today are still built by humans for humans, in a world increasingly driven by machine autonomy. For example, when designing complex systems such as aircraft, various regulatory standards must be complied with, which may require many different models and simulations (and consequently, the use of many different digital engineering tools) for evaluation. Today, such endeavors require the collaboration of numerous highly specialized subject matter experts who examine many regulatory standard documents, inevitably involving many slow and costly human steps in the design and engineering process. Furthermore, current certification processes typically require the manufacture of a physical manifestation of the digitally engineered component and / or system for evaluation in the physical world (e.g., for physical testing), which can slow down the iterative design and engineering process.

[0011] The interconnected digital engineering and certification ecosystems described herein (sometimes referred to as the “digital engineering metaverse”) address each of these issues and others. In particular, the interconnected digital engineering and certification ecosystems may include computing systems (e.g., including networked centralized or distributed computing subsystems or components) that interface with various centralized or distributed digital engineering tools (e.g., via application programming interfaces (APIs) and / or software development kits (SDKs)), where those digital engineering tools can be separate from the computing system or can be considered part of the computing system itself. Digital engineering tools can interface with APIs, and / or SDKs may enable users of the ecosystem (including providers of digital engineering tools) to develop their own APIs for their tools or models to enable their tools or models to interact with the system. For example, a new company might create a new MBSE tool and then use an SDK to add its tool to the ecosystem, thereby enabling that tool to automatically interoperate with other tools in the ecosystem via APIs. Furthermore, the new company may be able to maintain its APIs over time, so that the administrator of the entire ecosystem does not have to maintain all the different APIs for all the different tools. This architecture can have the advantage of increasing the ease of interoperability between digital engineering tools.For example, rather than requiring each individual digital engineering tool to integrate with every other individual digital engineering tool in the ecosystem, a computing system can enable the interoperable use of multiple digital engineering tools implemented in multiple other computing systems (or, in some cases, within the same computing system), as long as each tool is integrated with the computing system. Furthermore, rather than requiring users of digital engineering tools to interact separately with various digital engineering tools to perform modeling and simulation, a computing system can enable users to interact with and utilize a single user interface for the computing system in the ecosystem, and furthermore, the computing system in the ecosystem can interface with many digital engineering tools. This can result in a flatter learning curve for users, who only need to be familiar with a single user interface (e.g., the user interface associated with the computing system) rather than several different user interfaces (e.g., associated with various digital engineering tools). Also, this can increase the number of interfaces between digital engineering tools to N. 2 The computational complexity can be simplified from n to N, where N represents the number of digital engineering tools included in the ecosystem. This, in turn, can easily generate scalable applications, automation, and / or machine learning and artificial intelligence that span various digital engineering tools.

[0012] An interconnected digital engineering and certification ecosystem also has the advantage of including digitized regulatory and certification standards, compliance, computation, and testing (for example, for the development, testing, and certification of products and / or solutions), which can enable users to directly integrate relevant regulatory and certification standards, compliance, computation, and testing data into their digital engineering workflows. Regulatory and certification standards, compliance, computation, and testing may be referred to herein as "common validation and verification (V&V) products." In some implementations, the ecosystem's computing systems can interface with regulatory and / or certification authorities (for example, via a website operated by the authority) to retrieve digitized common V&V products published by the authority that may be critical to the products the user is designing. In some implementations, users can upload digitized common V&V products to the ecosystem themselves. Including digitized common V&V products in the ecosystem can be particularly beneficial for completing complex system engineering projects where many regulatory requirements may need to be met using several different digital engineering tools. By integrating both digital engineering tools with digitized common V&V products, the entire product design and engineering process (or parts thereof) can be digitized, eliminating or reducing time-consuming and costly steps such as human review of regulatory standards to identify regulatory requirements, human determination of which digital engineering tools are needed, and human assessment of whether regulatory requirements are met.For example, a computing system in a digital engineering and certification ecosystem may be configured to process regulatory and / or certification data corresponding to a digitized common V&V product, as well as engineering-related data output received from one or more digital engineering tools, to automatically assess whether one or more specified regulatory and / or certification requirements are met in the common V&V product. The computing system can generate reports, which can be presented to the user in an easy-to-read format and may even include recommendations for improvements to the user's digital prototype of the product (e.g., to meet unsatisfied regulatory and / or certification requirements). Importantly, all of this can be done without physical testing, without requiring any physical manifestation of the manufactured product. As digital models and simulations continue to become increasingly high-fidelity, certification of products such as unmanned aerial vehicles or other aircraft can also be performed digitally, saving time, cost, and materials associated with the physical evaluation and certification of the product. Throughout this specification, unmanned aerial vehicles and other aircraft are referred to as exemplary products, but the ecosystem can be developed using digital engineering tools and / or can be readily used for the design, engineering, testing, and / or certification of any product or solution (e.g., automobiles, pharmaceuticals, medical devices, processes, etc.) subject to regulatory and / or certification requirements.

[0013] An interconnected digital engineering and certification ecosystem also has the advantage of providing a single computing system (which may be centralized or distributed) through which various types of data flow throughout the entire design, engineering, testing, and / or certification process. For example, data on prototypes, common V&V products, the use of digital engineering tools to meet specific common V&V products, the success or failure of specific models and simulations, and various iterations of product designs can all be configured to flow securely through the ecosystem's computing systems (e.g., using zero-trust security) and corroborated by the ecosystem's computing systems. In some implementations, this data can be tracked and stored. This stored data can be audited for various purposes (e.g., to prevent security breaches or to perform data quality control). Stored data can also be explored to identify patterns in the data (e.g., using machine learning engines). For example, patterns in stored data can be used to determine which digital engineering tools are most useful to meet specific regulatory requirements after extensive use of the digital engineering and certification ecosystem by experts in the subject, to suggest adjustments to inputs or parameters to effectively run models and simulations, to perform sensitivity analyses for specific designs, and to design or partially design systems using machine learning and artificial intelligence. This can have the advantage of making the digital engineering and certification ecosystem increasingly user-friendly for non-experts in the subject, as it can be supported by computing systems throughout the entire design and engineering process based on data collected from more expert and / or experienced users, thereby accelerating the entire engineering and certification process.

[0014] An interconnected digital engineering and certification ecosystem may have the added benefit of enabling the development of a repository of previously evaluated designs and / or solutions related to one or more common V&V products that can be easily reused with minimal additional engineering effort. Such designs and / or solutions can be offered to users (e.g., both human and artificial intelligence users) for use as is or as a starting point for modification, thereby reducing redundant work and streamlining the design, engineering, testing, and certification processes. In some implementations, the repository may be searchable by users to identify previous designs and / or solutions generated by others. In some implementations, the repository (or specific elements within the repository) may also be specific to users with specific credentials (e.g., users associated with a particular company, team, technology field, etc.) to avoid disclosing confidential information while still facilitating effective collaboration. In some cases, user credentials may be used additionally or alternatively in an interconnected digital engineering and certification ecosystem for other purposes, such as coordinating the types of digital engineering tools (or functions within digital engineering tools) that users may access. For example, user credentials may correspond to the user's skill level and may be checked to ensure that the user is not overwhelmed by the functionality of digital engineering tools, which may exceed the user's skill set in order to use them effectively.

[0015] An interconnected digital engineering and certification ecosystem can offer further advantages, such as enabling the sharing of high-value digital engineering models while continuing to protect the intellectual property contained within the models. Many recent technology development projects involve multiple stakeholders working together (e.g., customers, prime integrators, suppliers, etc.) and require access to each other's models, but with different access permissions to the data. This system allows for precise specification of exactly which data within a model should be shared with each individual stakeholder, without revealing it to all data stakeholders. This selective sharing of information enables measurement and tracking of which data is consumed by each stakeholder (e.g., sharing only the inputs and outputs of a hydrodynamic pressure model) and how much data is consumed (e.g., how many times the hydrodynamic model is run). This measurement and tracking enables new business models based on the creation of models and data that can be monitored and monetized. In some implementations, this measurement and tracking can extend beyond the initial sharing of data to measuring and / or tracking subsequent or derivative uses of the data by third parties who were not involved in the initial sharing agreement. For example, a prime contractor might share data with a first government agency, and that first government agency could freely share data with a second government agency, and the prime contractor might have the ability to allow / not allow, track, and potentially monetize this further sharing. Such an implementation could have the advantage of enabling extremely granular capture of model data.

[0016] Other features and advantages of the description will become apparent from the following description and claims. Unless otherwise defined, technical and scientific terms used herein have the same meanings as commonly understood by an ordinary person skilled in the art to which the invention pertains.

[0017] A general aspect of this disclosure may include a computer-based method that includes the step of receiving design and / or engineering data (D / E data) corresponding to a prototype representation of a product from a user device. The method may also include the step of sending one or more inputs derived from the D / E data to one or more digital engineering tools for processing. The method may also include the step of receiving engineering-related data output from one or more digital engineering tools. The method may also include the step of receiving data corresponding to one or more common validation and verification (V&V) products related to the product. The method may also include the step of identifying one or more requirements of the product based on the data corresponding to one or more common V&V products. The method may also include the step of determining whether one or more requirements have been met based on the engineering-related data output and the data corresponding to one or more common V&V products. The method may also include the step of presenting information on the user device that corresponds to the engineering-related data output and / or the data corresponding to one or more common V&V products. The presented information may include an indication of whether one or more requirements have been met, or an indication of the probability that one or more requirements are met by the prototype representation of the product. The method includes the step of presenting information on the user device corresponding to engineering-related data output and / or data corresponding to one or more common V&V products, and then receiving instructions from the user device. In some implementations, the instructions correspond to one or more user interactions with the user device. The method also includes the step of performing one or more operations on D / E data in response to receiving instructions from the user device.

[0018] In some implementations, data corresponding to one or more common V&V products may be received from the user device. In some implementations, data corresponding to one or more common V&V products may be received from regulatory and / or certification authorities. In some implementations, the product may be an aircraft. In some implementations, at least a subset of one or more digital engineering tools may include model-based systems engineering (MBSE) tools, augmented reality (AR) tools, computer-aided design (CAD) tools, data analysis tools, modeling and simulation (M&S) tools, product lifecycle management (PLM) tools, simulation engines, requirements models, electronics models, test planning models, cost models, scheduling models, software modeling, supply chain models, manufacturing models, cybersecurity models, multi-attribute trade space tools, or mission effectiveness models. In some implementations, the step of determining whether one or more requirements have been met based on engineering-related data output may include determining whether one or more requirements have been met without any human input. In some implementations, the information presented may include recommended actions that a user of the user device may take to meet one or more requirements. In some implementations, recommended actions may include suggesting the use of a specific digital engineering tool of one or more digital engineering tools, suggesting the modification of one or more inputs sent to one or more digital engineering tools, suggesting the modification of one or more components of a prototype representation of a product, and / or suggesting the replacement of one or more components of a prototype representation of a product with a previously designed solution. In some implementations, the step of performing one or more operations on D / E data may include modifying the D / E data and / or deriving modified inputs from the D / E data for submission to one or more digital engineering tools.In some implementations, the computer-based method may include storing in a storage device usage data representing received data corresponding to one or more common V&V products, received D / E data, engineering-related data output from one or more digital engineering tools, indications of whether one or more requirements are met, indications of the probability that one or more requirements are met by a prototype representation of the product, one or more user interactions with a user device, and / or one or more operations on the D / E data. In some implementations, the computer-based method may include incorporating at least a portion of the usage data into a training dataset. The implementation may also include training a machine learning model based on the training dataset. In some implementations, a machine learning model may be configured to take as input information about another product designed by another user and to output suggestions for the other user to use one or more digital engineering tools, suggestions to modify one or more inputs sent to one or more digital engineering tools by another user, suggestions to modify one or more components of another prototype representation related to another user, suggestions to replace one or more components of another prototype representation with a previously designed solution, and / or suggestions for a completely or partially new design generated using a machine learning engine. In some implementations, a computer-based method may include a step of using stored usage data for one or more sensitivity analyses. In some implementations, a computer-based method may include a step of checking one or more user credentials before performing one or more operations on the D / E data. The implementation may also include a step of determining, based on one or more credentials, whether the user is likely to be qualified to perform one or more operations on the D / E data.

[0019] Another general aspect of this disclosure may include a system comprising memory for storing instructions that may be executable, and one or more processing devices coupled to the memory, configured to execute instructions in order to perform an operation. The operation includes receiving design and / or engineering data (D / E data) corresponding to a prototype representation of a product from a user device. The operation also includes sending one or more inputs derived from the D / E data to one or more digital engineering tools for processing. The operation also includes receiving engineering-related data output from one or more digital engineering tools. The operation also includes receiving data corresponding to one or more common validation and verification (V&V) products related to the product. The operation also includes identifying one or more requirements of the product based on the data corresponding to one or more common V&V products. The operation also includes determining whether one or more requirements have been met based on the engineering-related data output and the data corresponding to one or more common V&V products. The operation also includes presenting information on the user device corresponding to the engineering-related data output and / or the data corresponding to one or more common V&V products. The information presented may include indications of whether one or more requirements are met, or indications of the probability that one or more requirements are met by the product prototype representation. In some implementations, the operation may also include receiving commands from the user device after presenting information on the user device corresponding to engineering-related data output and / or data corresponding to one or more common V&V products, where the commands correspond to one or more user interactions with the user device. The operation may also include performing one or more operations on D / E data in response to receiving commands from the user device.

[0020] In some implementations, data corresponding to one or more common V&V products may be received from the user device. In some implementations, data corresponding to one or more common V&V products may be received from regulatory and / or certification authorities. In some implementations, the product may be an aircraft. In some implementations, at least a subset of one or more digital engineering tools may include model-based systems engineering (MBSE) tools, augmented reality (AR) tools, computer-aided design (CAD) tools, data analysis tools, modeling and simulation (M&S) tools, product lifecycle management (PLM) tools, simulation engines, requirements models, electronics models, test planning models, cost models, scheduling models, software modeling, supply chain models, manufacturing models, cybersecurity models, multi-attribute trade space tools, or mission effectiveness models. In some implementations, the step of determining whether one or more requirements have been met based on engineering-related data output may include determining whether one or more requirements have been met without any human input. In some implementations, the information presented may include recommended actions that a user of the user device may take to meet one or more requirements. In some implementations, recommended actions may include suggesting the use of a specific digital engineering tool of one or more digital engineering tools, suggesting the modification of one or more inputs sent to one or more digital engineering tools, suggesting the modification of one or more components of a prototype representation of a product, and / or suggesting the replacement of one or more components of a prototype representation of a product with a previously designed solution. In some implementations, performing one or more operations on D / E data may include modifying D / E data and / or deriving modified inputs from D / E data for submission to one or more digital engineering tools.In some implementations, the operation may include storing usage data in a storage device representing received data corresponding to one or more common V&V products, received D / E data, engineering-related data output from one or more digital engineering tools, indications of whether one or more requirements have been met, indications of the probability that one or more requirements are met by a prototype representation of the product, one or more user interactions with a user device, and / or one or more operations on the D / E data. In some implementations, the operation may include incorporating at least a portion of the usage data into a training dataset. In some implementations, the operation may also include training a machine learning model based on the training dataset. In some implementations, the machine learning model may be configured to receive as input information about another product being designed by another user and to output suggestions for the other user to use a specific digital engineering tool from one or more digital engineering tools, suggestions to modify one or more inputs sent by the other user to one or more digital engineering tools, suggestions to modify one or more components of another prototype representation related to another user, suggestions to replace one or more components of another prototype representation with a previously designed solution, and / or suggestions for a completely or partially new design generated using a machine learning engine. In some implementations, the operation may include using stored usage data for one or more sensitivity analyses. In some implementations, the operation may include checking one or more user credentials before performing one or more operations on the D / E data. In some implementations, the operation may also include determining, based on one or more credentials, whether the user is likely qualified to perform one or more operations on the D / E data.

[0021] Another general aspect of this disclosure may include one or more non-temporary machine-readable storage media that store instructions to be executed to perform an operation. The operation includes receiving design and / or engineering data (D / E data) corresponding to a prototype representation of the product from a user device. The operation also includes transmitting one or more inputs derived from the D / E data to one or more digital engineering tools for processing. The operation also includes receiving engineering-related data output from one or more digital engineering tools. The operation also includes receiving data corresponding to one or more common validation and verification (V&V) products related to the product. The operation also includes identifying one or more requirements of the product based on the data corresponding to one or more common V&V products. The operation also includes determining whether one or more requirements have been met based on the engineering-related data output and the data corresponding to one or more common V&V products. The operation also includes presenting information on the user device corresponding to the engineering-related data output and / or the data corresponding to one or more common V&V products. The information presented may include indications of whether one or more requirements are met, or indications of the probability that one or more requirements are met by the product prototype representation. In some implementations, the operation may also include receiving commands from the user device after presenting information on the user device corresponding to engineering-related data output and / or data corresponding to one or more common V&V products, where the commands correspond to one or more user interactions with the user device. The operation may also include performing one or more operations on D / E data in response to receiving commands from the user device.

[0022] In some implementations, data corresponding to one or more common V&V products may be received from the user device. In some implementations, data corresponding to one or more common V&V products may be received from regulatory and / or certification authorities. In some implementations, the product may be an aircraft. In some implementations, at least a subset of one or more digital engineering tools may include model-based systems engineering (MBSE) tools, augmented reality (AR) tools, computer-aided design (CAD) tools, data analysis tools, modeling and simulation (M&S) tools, product lifecycle management (PLM) tools, simulation engines, requirements models, electronics models, test planning models, cost models, scheduling models, software modeling, supply chain models, manufacturing models, cybersecurity models, multi-attribute trade space tools, or mission effectiveness models. In some implementations, determining whether one or more requirements have been met based on engineering-related data output may include determining whether one or more requirements have been met without any human input. In some implementations, the information presented may include recommended actions that a user of the user device may take to meet one or more requirements. In some implementations, recommended actions may include suggesting the use of a specific digital engineering tool of one or more digital engineering tools, suggesting the modification of one or more inputs sent to one or more digital engineering tools, suggesting the modification of one or more components of a prototype representation of a product, and / or suggesting the replacement of one or more components of a prototype representation of a product with a previously designed solution. In some implementations, performing one or more operations on D / E data may include modifying D / E data and / or deriving modified inputs from D / E data for submission to one or more digital engineering tools.In some implementations, the operation may include storing usage data in a storage device representing received data corresponding to one or more common V&V products, received D / E data, engineering-related data output from one or more digital engineering tools, indications of whether one or more requirements have been met, indications of the probability that one or more requirements are met by a prototype representation of the product, one or more user interactions with a user device, and / or one or more operations on the D / E data. In some implementations, the operation may include incorporating at least a portion of the usage data into a training dataset. In some implementations, the operation may also include training a machine learning model based on the training dataset. In some implementations, the machine learning model may be configured to receive as input information about another product being designed by another user and to output suggestions for the other user to use a specific digital engineering tool from one or more digital engineering tools, suggestions to modify one or more inputs sent by the other user to one or more digital engineering tools, suggestions to modify one or more components of another prototype representation related to another user, suggestions to replace one or more components of another prototype representation with a previously designed solution, and / or suggestions for a completely or partially new design generated using a machine learning engine. In some implementations, the operation may include using stored usage data for one or more sensitivity analyses. In some implementations, the operation may include checking one or more user credentials before performing one or more operations on the D / E data. In some implementations, the operation may also include determining, based on one or more credentials, whether the user is likely qualified to perform one or more operations on the D / E data. [Brief explanation of the drawing]

[0023] [Figure 1]A diagram showing an exemplary interconnected digital engineering and authentication ecosystem, as well as digitally authenticated products. [Figure 2A] A flowchart showing an exemplary workflow including an interconnected digital engineering and authentication ecosystem. [Figure 2B] A flowchart showing an exemplary workflow including an interconnected digital engineering and authentication ecosystem. [Figure 3] A diagram showing a series of exemplary displays presented on a user device corresponding to the exemplary workflow of FIGS. 2A - 2B. [Figure 4] A flowchart showing an exemplary product design process using an interconnected digital engineering and authentication ecosystem. [Figure 5] A diagram showing how an interconnected digital engineering and authentication ecosystem can be monetized. [Figure 6] A flowchart showing a process for product development executed by a computing system of an interconnected digital engineering and authentication ecosystem. [Figure 7] A diagram showing an example of a computing environment. [Figure 8] A diagram showing an exemplary architecture of an interconnected digital engineering and authentication ecosystem.

Best Mode for Carrying Out the Invention

[0024] This disclosure describes an interconnected digital engineering and certification ecosystem that can enable new capabilities and improve processes for digital product development, including digital design, digital engineering, digital testing, and digital certification of products. For the purposes of this disclosure, the terms “design” and “engineer” are used substantially synonymously and are broadly defined to encompass the process of intelligently developing a product to solve a particular problem (e.g., to improve performance, enhance aesthetic appeal, meet one or more regulatory requirements).

[0025] Figure 1 shows an exemplary interconnected digital engineering and certification ecosystem 100 and examples of digitally certified products 112A–112C (collectively referred to as digitally certified products 112). For example, in some implementations, digitally certified product 112A may be an unmanned aerial vehicle [UAV] or other aircraft, digitally certified product 112B may be a pharmaceutical or other chemical or biological compound, and digitally certified product 112C may be a process, such as a manufacturing process. Generally, digitally certified products 112 may include any product, process, or solution that can be developed (partially or entirely) using digital engineering tools (e.g., digital engineering tool 102). In some implementations, digitally certified products 112 may not be limited to physical products and may also include non-physical products (e.g., processes, software, etc.). While physical and physically interacting systems often require multiple digital engineering tools simply for M&S needs to assess compliance with common V&V products, many complex non-physical systems may also require multiple digital engineering tools for product development, testing, and / or certification. With this in mind, various other possibilities regarding digitally certified products will be recognized by those skilled in the art.

[0026] A digitally certified product 112 may be designed and / or certified using an interconnected digital engineering and certification ecosystem 100. The interconnected digital engineering and certification ecosystem 100 may include user devices 106A or APIs (or other similar machine-to-machine communication interfaces) 106B operated by users (e.g., human users 104A of varying skill levels, or artificial users 104B such as algorithms, artificial intelligence, or other software), and a computing system 108 connected to (and / or including) a data storage unit 118, a machine learning engine 120, and an application and service layer 122. For the purposes of clarity, all users selected from various potential human users 104A or artificial users 104B are simply referred to herein as user 104. In some implementations, the computing system 108 may be a centralized computing system, while in other implementations, the computing system 108 may be a distributed computing system. In some cases, user 104 can be considered part of ecosystem 100, while in other embodiments, user 104 can be considered separate from ecosystem 100.Ecosystem 100 includes one or more digital engineering tools 102 (e.g., data analysis tools 102A, CAD and finite element analysis tools 102B, simulation tools 102C, pharmaceutical M&S tools 102D-102E, manufacturing M&S tools 102F-102G, etc.) and common V&V products 110 (e.g., regulatory standards 110A-110F related to UAV development and certification, medical standards 110G (e.g., CE marking (Europe), FCC Declaration of Conformity (USA), IECEE CB scheme (Europe, North America, parts of Asia & Australia), CDSCO (India), FDA (USA), etc.)), medical certification regulations 110H (e.g., ISO 13485, ISO 14971, ISO 9001, ISO 62304, ISO 10993, ISO 15223, ISO 11135, ISO 11137, ISO This further includes repositories of standards such as 11607, IEC 60601, etc., manufacturing standards 110I (e.g., ISO 9001, ISO 9013, ISO 10204, EN 1090, ISO 14004, etc.), and manufacturing certification regulations 110J (e.g., General Certification of Conformity (GCC), etc.).

[0027] The computing system 108 of ecosystem 100 is centrally located within the architecture of ecosystem 100 and is configured to communicate with user devices 106A or API 106B (e.g., APIs associated with artificial users 104B), digital engineering tools 102 (e.g., via application programming interfaces [APIs] / software development kits [SDKs] 114), and the repository of common V&V products 110 (e.g., via API / SDK 116) (e.g., to receive data from them and send data to them). For example, computing system 108 may be configured to communicate with user devices 106A and / or API 106B to send or receive data corresponding to design prototypes, information about users (e.g., user credentials), engineering-related inputs / outputs associated with digital engineering tools 102, digitized common V&V products, product design evaluations, user instructions (e.g., search requests, data processing instructions, etc.), and various other things. The computing system 108 may also be configured to communicate with one or more digital engineering tools 102 to transmit engineering-related inputs for performing analyses, models, simulations, tests, etc., and to receive engineering-related outputs related to the results. The computing system 108 may also be configured to communicate with the repository of common V&V products 110 to retrieve data corresponding to one or more digitized common V&V products 110 and / or upload new common V&V products (e.g., new common V&V products received from user 104) to the repository of common V&V products 110. All communications may be transmitted and backed up securely, for example, using methods that rely on zero-trust security.

[0028] The computing system 108 can process and / or store data it receives, and in some implementations (for example, using storage 118), it can access a machine learning engine 120 and / or an application and service layer 122 (either included as part of the computing system 108 or located outside of the computing system 108) to identify useful insights based on the data, as further described herein. The centralized placement of the computing system 108 within the architecture of the ecosystem 100 has many advantages, including reducing the technical complexity of integrating various digital engineering tools 102, enhancing the user 104's product development experience, intelligently linking common V&V products (e.g., criteria 110A-110F) to the digital engineering tools 102 most useful for meeting the requirements associated with the common V&V products, and enabling monitoring, storage, and analysis of various data flowing between elements of the ecosystem 100 throughout the entire product development process. In some implementations, data flowing through (and potentially stored by) computing system 108 can also be auditable for purposes such as preventing security breaches or performing data quality control.

[0029] Referring to one specific example shown in Figure 1, user 104 can manufacture a digitally certified UAV 112B using the digital engineering and certification ecosystem 100. For example, user 104 may be primarily interested in certifying the UAV as meeting the requirements of a specific regulatory standard 110E (e.g., "MIL-HDBK 516C 4.1.4 -- Failure Conditions") regarding the failure condition of the UAV. In this use scenario, user 104 can develop a digital prototype of the UAV on user device 106A or using API 106B, and send the prototype data (e.g., as at least one of a CAD file, MBSE file, etc.) to computing system 108. Along with the prototype data, user 104 may transmit additional data via user device 106A, including indications of common V&V products (e.g., regulatory standard 110E) that user 104 is interested in certifying the product, user credential information for accessing one or more capabilities of computing system 108, and / or instructions for running one or more digital models, tests, and / or simulations using a subset of digital engineering tools 102.

[0030] Referring to another example shown in Figure 1, user 104 can use the digital engineering and certification ecosystem 100 to manufacture digitally certified pharmaceuticals, chemical compounds, or biological agents 112A. For example, user 104 may be primarily interested in certifying pharmaceuticals, chemical compounds, or biological agents 112A as meeting the requirements of specific medical standards 110G and medical certification regulations 110H. In this use scenario, user 104 can develop a digital prototype of the pharmaceutical, chemical compound, or biological agent on user device 106A or using API 106B, and send the prototype data (for example, as a molecular modeling file) to computing system 108. Along with the prototype data, user 104 may transmit additional data via user device 106A, including indications of common V&V products (e.g., medical standards 110G and medical certification regulations 110H) for which user 104 is interested in certifying the product, user credential information for accessing one or more capabilities of computing system 108, and / or instructions for running one or more digital models, tests, and / or simulations using a subset of digital engineering tools 102 (e.g., pharmaceutical M&S tools 102D-102E).

[0031] Referring to yet another specific example shown in Figure 1, user 104 can use the digital engineering and certification ecosystem 100 to manufacture a digitally certified manufacturing process 112C. For example, user 104 may be primarily interested in certifying the manufacturing process 112C as meeting the requirements of a specific manufacturing standard 110I and manufacturing certification regulation 110J. In this use case, user 104 can develop a digital prototype of the manufacturing process on user device 106A or using API 106B and send the prototype data to computing system 108. Along with the prototype data, user 104 may transmit additional data via user device 106A, including indications of common V&V products (e.g., manufacturing standard 110I and manufacturing certification regulation 110J) that user 104 is interested in authenticating the process, user credential information for accessing one or more capabilities of computing system 108, and / or instructions for running one or more digital models, tests, and / or simulations using a subset of digital engineering tools 102 (e.g., manufacturing M&S tools 102F-102G).

[0032] In any of the above examples, the computing system 108 can receive data transmitted from the user device 106A and / or API 106B, process the data, and evaluate whether the common V&V products of interest (e.g., regulatory standard 110E, medical standard 110G, medical certification regulation 110H, manufacturing standard 110I, manufacturing certification regulation 110J, etc.) are met by the user's digital prototype. For example, this may include communicating with a repository of common V&V products 110 (via API / SDK 116) to retrieve relevant common V&V products of interest, and processing regulatory and / or certification data related to the common V&V products to identify one or more requirements for a UAV prototype, pharmaceuticals, chemical compounds, or biological drug prototype, manufacturing process prototype, etc. In some implementations, the repository for common V&V product 110 may be hosted by a regulatory and / or certification authority (or another third party), and retrieving regulatory and / or certification data may involve using API / SDK 116 to interface with one or more data resources maintained by the regulatory and / or certification authority (or another third party). In some implementations, regulatory and / or certification data may be provided directly by user 104 via user device 106A and / or API 106B (for example, together with prototype data).

[0033] Evaluating whether a common V&V product of interest (e.g., regulatory standard 110E, medical standard 110G, medical certification regulation 110H, manufacturing standard 110I, manufacturing certification regulation 110J, etc.) is met by the user's digital prototype may also include processing prototype data received from the user device 106A or API 106B to determine whether one or more identified requirements are actually met. In some implementations, the computing system 108 may include one or more plug-ins, local applications, etc., for direct processing of the prototype data within the computing system 108. In some implementations, the computing system may simply preprocess the received prototype data (e.g., to derive inputs for the digital engineering tool 102) and then send instructions and / or input data to a subset of the digital engineering tool 102 via API / SDK 114 for further processing.

[0034] Not all digital engineering tools 102 are necessarily required to meet specific regulatory and / or certification standards. Therefore, in the example of a UAV given in Figure 1, the computing system 108 may determine that only data analysis tool 102A and finite element analysis tool 102B are required to meet the regulatory standard 110E regarding failure conditions. In the example of a drug, chemical compound, or biological agent given in Figure 1, the computing system 108 may determine that only drug M&S tools 102D-102E are required to meet medical standard 110G and medical certification regulation 110H. In the example of a manufacturing process given in Figure 1, the computing system 108 may determine that only manufacturing M&S tools 102F-102G are required to meet manufacturing standard 110I and manufacturing certification regulation 110J. In other implementations, user 104 may, provided that user 104 is a qualified subject matter expert, identify specific subsets of digital engineering tools 102B that should be used to meet common V&V products of interest. In other implementations, user 104 may input several proposed digital engineering tools 102 into computing system 108 to satisfy common V&V products of interest, and computing system 108 may recommend a modified subset of digital engineering tools 102 to user 104 for final approval, provided that user 104 is a qualified subject matter expert. After the subset of digital engineering tools 102 has been identified, computing system 108 may send instructions and / or input data to the identified subset of digital engineering tools 102 to run one or more models, tests, and / or simulations. The results of these models, tests, and / or simulations (or "engineering-related data outputs") may be sent back and received by computing system 108.

[0035] In other implementations, user 104 may input a digital engineering tool (e.g., digital engineering tool 102F) required to satisfy common V&V product 110I, and computing system 108 may determine that another digital engineering tool (e.g., digital engineering tool 102G) is also required to satisfy common V&V product 110I. The computing system can then send instructions and / or input data to both digital engineering tools (e.g., digital engineering tools 102F and 102G), and the outputs of these digital engineering tools may be transmitted and received in computing system 108. In some cases, input data sent to one of the digital engineering tools (e.g., digital engineering tool 102G) may be derived (e.g., by computing system 108) from the output of another digital engineering tool (e.g., digital engineering tool 102F).

[0036] After receiving engineering-related data output from the digital engineering tool 102, the computing system 108 can process the received engineering-related data output to evaluate whether the requirements identified in the common V&V product of interest (e.g., regulatory standard 110E, medical standard 110G, medical certification regulation 110H, manufacturing standard 110I, manufacturing certification regulation 110J, etc.) are met. In some implementations, the computing system 108 can generate a report summarizing the evaluation results and send the report to the user device 106A or API 106B for review by the user 104. If all requirements are met, the prototype can be certified, resulting in a digitally certified product 112 (e.g., digitally certified pharmaceutical, chemical compound, or biological agent 112A, digitally certified UAV 112B, digitally certified manufacturing process 112C, etc.). However, if some regulatory requirements are not met, additional steps may need to be taken by the user 104 to certify the product prototype. In some implementations, reports sent to the user may include recommendations regarding these additional steps (e.g., suggesting one or more design changes, suggesting replacing one or more components with previously designed solutions, suggesting one or more adjustments to the inputs for models, tests, and / or simulations). If the requirements for the common V&V product are partially met or exceed the collective capabilities of the distributed engineering tools 102, the computing system 108 may provide user 104 with a report recommending partial certification, compliance, or fulfillment of a subset of the common V&V product (e.g., digital certification of a subsystem or subprocess of a prototype). The process for generating recommendations for user 104 is described in more detail below.

[0037] In response to reviewing the report, user 104 can make design changes to the digital prototype locally and / or send one or more instructions to computing system 108 via user device 106A or API 106B. These instructions may include, for example, instructions to computing system 108 to re-evaluate the updated prototype design, use one or more different digital engineering tools 102 for the evaluation process, and / or modify the inputs to the digital engineering tools 102. Furthermore, computing system 108 can receive the user's instructions, perform one or more additional data operations in accordance with these instructions, and provide user 104 with an updated report. Through this iterative process, user 104 can leverage the interconnected digital engineering and certification ecosystem 100 to design prototypes (e.g., UAV prototypes, pharmaceutical prototypes, manufacturing process prototypes, etc.) and ultimately certify them with respect to common V&V products of interest (e.g., by providing certification compliance information). Importantly, since all of these steps are performed in the digital world (e.g., using digital prototypes, digital models / tests / simulations, and digital certifications), a significant amount of time, cost, and material can be saved compared to processes that involve physical prototyping, evaluation, and / or certification, such as similar UAVs, pharmaceuticals, and manufacturing processes. If the requirements related to the common V&V product are partially met or exceed the collective capabilities of the digital engineering tools 102, the computing system 108 may provide user 104 with a report recommending partial certification, compliance, or fulfillment of a subset of the common V&V product (e.g., digital certification of a subsystem or subprocess of a prototype).

[0038] While the above example focuses on the use of the interconnected digital engineering and certification ecosystem 100 by a single user, further benefits of the ecosystem 100 can be realized through repeated use of the ecosystem 100 by multiple users. As mentioned above, the centralized positioning of the computing system 108 within the architecture of the ecosystem 100 allows the computing system 108 to monitor and remember the various data flows through the ecosystem 100. Thus, as an increasing number of users utilize the ecosystem 100 for digital product development, the data associated with each use of the ecosystem 100 can be stored (for example, in storage 118) and analyzed to yield various insights, which can then be used to further automate the digital product development process and make it more navigable for non-experts.

[0039] In fact, in some implementations, user credentials for user 104 can indicate user 104's skill level, allowing the user to control the amount of automated assistance provided. For example, a non-expert in the target group may only be allowed to use the ecosystem 100 to browse pre-built designs and / or solutions, use digital engineering tools 102 with specific default parameters, and / or follow a predetermined workflow, with automated assistance guiding user 104 through the product development process. A more skilled user, on the other hand, may still be provided with automated assistance, but may be given more opportunities to override the default or suggested workflows and settings.

[0040] In some implementations, the computing system 108 may host applications and services 122 that automate or partially automate expected or common interface and / or data exchange, including components of a common V&V product, expected or common data transmission, including components of data transmission, from users 104, expected or common interface and / or data exchange, including components of interface, between various digital engineering tools 102, expected or common interface and / or data exchange, including components of interface, with machine learning models implemented on the computing system 108 (e.g., models trained and / or implemented by the machine learning engine 120), and expected or common interface and / or data exchange between applications and services themselves (e.g., within the application and service layer 122).

[0041] In some implementations, data from multiple uses of ecosystem 100 (or a portion thereof) may be aggregated to develop a training dataset. This training dataset may then be used to train a machine learning model (for example, using a machine learning engine 120) to perform a variety of tasks, including identifying which of the digital engineering tools 102 should be used to satisfy a particular common V&V product, identifying specific models, tests, and / or simulations (including their inputs) to be performed using the digital engineering tools 102, identifying common V&V products that need to be considered with respect to a particular type of product, identifying one or more recommended actions that a user 104 should take in response to unsatisfied regulatory requirements, and estimating the sensitivity of a model / test / simulation to a particular input. The output of a trained machine learning model may be used to implement various features of an interconnected digital engineering and certification ecosystem 100, including automatically suggesting inputs (e.g., inputs to a digital engineering tool 102) based on previously inputs, predicting time and cost requirements for developing a product, predictively estimating the results of sensitivity analysis, and even suggesting design changes, original designs, or design alternatives (e.g., by assistive or generative AI) to a user's prototype to overcome one or more requirements (e.g., regulatory and / or certification requirements) associated with a common V&V product. In some implementations, given sufficient training data, the machine learning engine 120 may independently generate new designs, models, simulations, tests, and / or common V&V products based on data collected from multiple uses of the ecosystem 100.

[0042] In addition to storing usage data to enable the development of machine learning models, previous prototype designs and / or solutions (e.g., previously designed components, systems, models, simulations, and / or other engineering representations thereof) may be stored within the ecosystem 100 (e.g., within storage 118) to enable users to search for and build upon the work of others. For example, previously designed components, systems, models, simulations, and / or other engineering representations thereof can be searched by user 104 and / or proposed to user 104 by computing system 108 to meet one or more requirements related to a common V&V product. Previously designed components, systems, models, simulations, and / or other engineering representations thereof can be used as is by user 104 or as a starting point for additional modifications. This store (repository) of previously designed components, systems, models, simulations, and / or other engineering representations thereof (whether or not they were ultimately certified) can be monetized to create a marketplace for digital products, which can be used to save time in the digital product development process, provide users with alternative design ideas, avoid redundant effort, and for various other purposes. In some implementations, data corresponding to previous designs and / or solutions may only be stored if the user who developed the design and / or solution chooses to share the data. In some implementations, the repository of previous designs and / or solutions may be containerized for private use (for example, to avoid the undesirable disclosure of extremely non-information) for private use within a single company, team, organization, or technology field.In some implementations, user credentials associated with user 104 can be checked by computing system 108 to determine which designs and / or solutions stored in the repository can be accessed by user 104. In some implementations, the use of previously designed components, systems, models, simulations, and / or other engineering representations thereof may be available only to other users who pay a license fee.

[0043] Referring here to Figures 2A and 2B, an example of an exemplary digital product development and certification workflow 200 that can be implemented using the integrated digital engineering and certification ecosystem 100 (shown in Figure 1) is provided. Although not intended to be restrictive, the workflow 200 is used to illustrate a descriptive and practical example of the kind of workflow made possible by the ecosystem 100 and its various features. In Figures 2A and 2B, the individual steps of the workflow 200 are grouped by elements of the ecosystem 100 that execute them (i.e., a user device 106A or API 106B operated by user 104, a computing system 108 connected to (and / or containing) storage 118, a machine learning engine 120, and an application and service layer 122, digital engineering tools 102, and a repository of common V&V products 110).

[0044] In step 202, user 104 can upload an MBSE file corresponding to the digital representation of the product (e.g., a UAV) from user device 106A or API 106B to computing system 108. In step 204, user 104 can also upload a CAD file corresponding to the digital representation of the product from user device 106 to computing system 108.

[0045] The computing system 108 can receive an MBSE file (206), process the MBSE file to extract weight requirements (208), and send the data to an MBSE tool (210). For example, the data sent to the MBSE tool may include updated weight data, and the computing system 108 can request the MBSE tool to update the MBSE file with that updated weight data. The request may be made, for example, via API 114 shown in Figure 1.

[0046] Similarly, the computing system 108 can receive a CAD file (212), process the CAD file to calculate, for example, the mass properties of a digital prototype (214), and send the data to a CAD tool (216). For example, the data sent to the CAD tool may include identified weight problems, and the computing system 108 can request the CAD tool to update the CAD file with those identified weight problems (for example, by highlighting the problems identified within the CAD file). This request can also be made, for example, via the API / SDK 114 shown in Figure 1.

[0047] In step 218, the MBSE tool (for example, one of the digital engineering tools 102) can receive data sent from the computing system 108 to the MBSE tool. The MBSE tool can then update the weight data in the MBSE file (220) and export the updated MBSE file to the computing system 108 (222).

[0048] Similarly, in step 224, a CAD tool (for example, another of the digital engineering tools 102) can receive data sent to the CAD tool from the computing system 108. The CAD tool can then update the CAD file to highlight issues within the CAD file (226) and export the updated CAD file to the computing system 108 (228).

[0049] In step 230, the computing system 108 can receive the updated CAD files and MBSE files exported by the CAD tool and MBSE tool, respectively.

[0050] In step 232, the computing system 108 may send a request for data corresponding to one or more common V&V products (e.g., regulatory and / or certification data) (232). For example, the request may be sent via the API / SDK 116 shown in Figure 1 so that it is processed in a repository of common V&V products 110, which may be off-the-shelf and / or hosted by a certification authority or another third party, as described above. Step 232 is shown in the workflow 200 after the communication of the computing system 108 with the digital engineering tool 102 (e.g., steps 210, 216, 222, 228, 230), but in some implementations, the computing system 108 may retrieve data corresponding to one or more common V&V products from the repository of common V&V products 110 before communication with the digital engineering tool 102. In the example shown in Figures 2A and 2B, the data requested in step 232 may correspond to “MIL HDBK 516c 5.5.2 (JSSG-2006)”—a regulatory standard that specifies the weight and center of gravity requirements for an aircraft to be certified as “airworthy.” The repository for common V&V product 110 receives a request for data corresponding to one or more common V&V products (236), transmits data corresponding to the computing system 108 (238), and the computing system 108 can then receive data corresponding to one or more common V&V products (e.g., regulatory and / or certification data) (240).

[0051] In step 242, the computing system 108 can process the updated CAD and MBSE files (received in step 230) and the data corresponding to one or more common V&V products (received in step 240) to identify and evaluate one or more certification requirements. For example, the computing system can automatically process the CAD and MBSE files to calculate the weight and center of gravity of the physical manifestation of the digital prototype. The computing device can then compare these to the aircraft weight and center of gravity requirements identified in the data corresponding to "MIL HDBK 516c 5.5.2 (JSSG-2006)".

[0052] In step 244, based on the computing system's evaluation of the requirements identified in the data corresponding to "MIL HDBK 516c 5.5.2 (JSSG-2006)", the computing system 108 may generate a report and send it to the user device 106A or API 106B. The report may summarize the evaluation results, including an indication of whether the identified requirements have been met. In some implementations, the report may also include one or more recommended actions for the user. The recommendations may be generated using the machine learning engine 120, for example, as previously described above in relation to Figure 1.

[0053] In step 246, the user device 106A or API 106B may receive a report, and user 104 may review the report. For example, the report may be presented on the display of user device 106A for review by human user 104A and / or received by API 106B for processing by artificial user 104B. Depending on the review of the report, user 104 may operate user device 106A or API 106B to update the prototype design and / or send data representing one or more user instructions to computing system 108 (248). As previously stated, such user instructions may include instructions to computing system 108 to re-evaluate the updated prototype design, use one or more different digital engineering tools 102 for the evaluation process, and / or modify the inputs to the digital engineering tools 102. In step 250, computing system 108 may receive one or more user instructions and perform one or more data operations in accordance with one or more user instructions. In this way, workflow 200 can enable iterative design of digital prototypes using an interconnected digital engineering and certification ecosystem 100 to design and certify products such as UAVs, pharmaceuticals, and manufacturing processes entirely within a digital world.

[0054] Referring next to Figure 3, a series of exemplary displays 300 shown on user device 106A are shown. In implementations including an artificial user 104B that interfaces with the computing system via API 106B, it should be noted that displays are not necessary, as the artificial user 104B can directly process the digital computer files received in API 106B without further visualization. The series of exemplary displays 300 may correspond to the exemplary workflow 200 described in relation to Figures 2A-2B. In this case as well, these displays are not intended to be restrictive, but merely to illustrate the kind of user experience that user 104 (and especially human user 104A) may encounter while using the interconnected digital engineering and certification ecosystem 100 for digital product development. The series of exemplary displays 300 described herein highlight the ease of use of the ecosystem 100 and the avoided complexity that would require users to interface separately with individual digital engineering tools and manually review complex common V&V products to evaluate whether a product prototype should be certified.

[0055] Display 302 shows a login screen that may be displayed on user device 106A. The login screen may prompt user 104 to enter user credentials (e.g., username and password) to access computing system 108 and the rest of the interconnected digital engineering and authentication ecosystem 100. User credentials associated with user 104 can perform various functions. For example, as mentioned above, user credentials can be associated with the user's skill level, which can control which functions of ecosystem 100 user 104 can access. In some implementations, user credentials can additionally or alternatively be associated with the user's affiliation (e.g., a specific company and / or organization), which can determine previously designed products and / or solutions that the user may search for and / or be offered by computing system 108. In general, user credentials can help ensure that user 104 can only access information within ecosystem 100 that user 104 is entitled to and / or authorized to access.

[0056] When user 104 logs in from user device 106A, user device 106A may be used to develop a digital prototype of a product. For example, display 304 shows a modeling screen that user 104 might see while developing a digital model of a UAV (for example, using a CAD tool). Once the prototype is developed, the user can upload prototype data, such as CAD files and / or MBSE files, to computing system 108 (for example, as in steps 202 and 204 of workflow 200). Thus, display 306 shows a screen that can prompt user 104 to upload MBSE files and CAD files to computing system 108.

[0057] When a user uploads MBSE and CAD files to the computing system 108, the computing system 108 can perform several steps (for example, steps 206, 207, 210, 212, 214, 216, 230, 232, 240, 242, 244 of workflow 200) to evaluate the prototype with respect to one or more requirements identified within the common V&V product and generate a report summarizing the evaluation. In doing so, the computing system 108 may communicate with the digital engineering tools 102 and the repository of the common V&V product 110, which themselves can perform actions (for example, steps 218, 220, 222, 224, 226, 228, 236, 238 of workflow 200) to facilitate the evaluation of the prototype. These steps may take some time to complete (for example, ranging from a few seconds to several hours), during which time display 308 may be shown on the screen of the user device 106A, providing information about the current status of the prototype evaluation.

[0058] Once the prototype evaluation is complete (for example, in step 244 of workflow 200), the generated report may be sent from computing system 108 to user device 106A. Display 310 shows the screen of user device 106A presenting the report to user 104. The report may present information indicating whether one or more requirements identified within the common V&V product of interest have been met, and may also present information about one or more issues that resulted in unsatisfied requirements (e.g., problematic components of the device). In some implementations, the information presented may also include more detailed data from the evaluation and / or proposed solutions to resolve one or more issues in order to meet the requirements. The easy-to-understand format of the report presented on Display 310 can help user 104 understand why the prototype may not meet one or more requirements and can provide user 104 with immediately actionable suggestions for improving the digital prototype. Even in implementations that include an artificial user 104B (where screen display is not required), a concise or standardized report in the form of a digital computer file sent to API 106B can similarly help the artificial user 104B understand why the prototype may not meet one or more requirements, and can provide the artificial user 104B with readily actionable suggestions for improving the digital prototype.

[0059] Referring next to Figure 4, a flowchart 400 is depicted illustrating an exemplary product design process using an interconnected digital engineering and certification ecosystem 100. By translating elements and properties such as physical laws from the physical world 402 to the digital world 406 via transfer function models and tools 404, product development and certification can be made entirely (or nearly entirely or partially) within the ecosystem 100. It should be noted that the entire physical world may not be perfectly replicated at once, and in some cases, a very specific subset of the physical world may be digitally described by any number of models or simulations (for example, a model describing turbulent air between 200 knots and 600 knots, complemented by another model describing turbulent air between 500 knots and 800 knots), and that these models or simulations together constitute a sufficient representation of the physical world to enable certification of a particular system or product. An interconnected digital engineering and certification ecosystem (e.g., ecosystem 100) can enable the digital representation of the physical world to be built in a modular and complementary manner, without requiring coordination and with coherent incentives (e.g., protection of intellectual property, monetization of models or valuable digital representations), allowing each of these parts to be individually added by many different people and many different entities, enabling each model or simulation to connect with and build upon the others. For example, digital product development 408, digital product testing 410, and digital product certification 412 can all take place within ecosystem 100, entirely within the digital realm or “metaverse” to produce a finalized digital product design 414. Following this product design process, the finalized digital product design only needs to be translated into the physical world (e.g., by manufacturing 416) at the very end of the product design process to manufacture the final product 418 in the physical world.This is in stark contrast to current product development and certification workflows, which utilize digital engineering tools but often still require the physical manufacturing, testing, and certification of prototypes throughout the iterative product development, product testing, and product certification process. Compared to such workflows, the product design process enabled by an interconnected digital engineering and certification ecosystem can therefore result in significant savings in time, cost, materials, and environmental impact.

[0060] Referring to Figure 5, the interconnected digital engineering and authentication ecosystem 100 provides various different opportunities for monetization (shown as blocks 500A-500D). In some implementations, the interaction between user 104 and computing system 108 may include monetization opportunity 500A. For example, user 104 may be charged for sending commands to computing system 108, and / or user 104 may be charged for downloading data (e.g., authentication reports) from computing system 108. The pricing can be subscription-based (e.g., charging a monthly or annual fee for using computing system 108), usage-based (e.g., charging user 104 based on the number of interactions with computing system 108, the amount of time spent interacting with computing system 108, etc.), or a hybrid (e.g., using a freemium model).

[0061] In some implementations, the interaction between the computing system 108 and the digital engineering tool 102 may include a monetization opportunity 500B. For example, a user 104 may be charged for transmitting data between the computing system 108 and / or the digital engineering tool 102. In some implementations, the fees paid by the user 104 may be split between the third-party provider of the digital engineering tool 102 and the operator of the computing system 108. In some implementations, the third-party providers of the digital engineering tool 102 themselves may pay a fee to the operator of the computing system 108 to include their digital engineering tools in the ecosystem 100. The user 104's fees can be subscription-based (for example, charging a monthly or annual fee to access a specific digital engineering tool 102), usage-based (for example, charging the user 104 based on the amount of data transferred between the digital engineering tool 102 and the computing system 108, the amount of processing time required by the digital engineering tool 102, etc.), or a hybrid (for example, using a freemium model).

[0062] In some implementations, the interaction between the computing system 108 and the repository of the common V&V product 110 may include a monetization opportunity 500C. For example, a user 104 may be charged for sending data between the computing system 108 and / or the repository of the common V&V product 110. In some implementations, the fees paid by user 104 may be split between the authority operating the repository of the common V&V product 110 and the operator of the computing system 108. The user 104's fees may be subscription-based (e.g., a monthly or annual fee charged for accessing the repository of the common V&V product 110), usage-based (e.g., user 104 is charged based on the amount of data transferred between the repository of the common V&V product 110 and the computing system 108, the number of common V&V products requested, etc.), or a hybrid (e.g., using a freemium model).

[0063] In some implementations, the final authentication of the digitally authenticated product 112 by the computing system 108 may also include a monetization opportunity 500D. For example, user 104 may be charged a fee for performing formal authentication of their product. Additionally or alternatively, user 104 may be charged a fee for downloading proof of authentication.

[0064] Referring now to Figure 8, an exemplary architecture of the Digital Engineering (DE) Platform 800 (including the interconnected Digital Engineering and Authentication Ecosystem 100) is shown. The exemplary architecture of the Digital Engineering Platform 800 is designed in accordance with zero-trust security principles and further designed to support scalability, as well as robust and resilient operation.

[0065] In one embodiment, the architecture of the digital engineering platform 800 includes several components, namely a digital engineering (DE) platform enclave 802, a cloud service 804, and a customer environment 810. The customer environment 810 optionally includes a DE platform exclave 816.

[0066] The DE Platform Enclave 802 can serve as the starting point for services provided by Platform 800. Enclave 802 can be visualized as a central command hub responsible for managing and functioning operations. For example, Enclave 802 can be implemented using the computer system 108 of the interconnected digital engineering and authentication ecosystem 100 described above. The DE Platform Enclave 802 acts as a centralized command and control hub responsible for orchestrating and managing the operations of all platforms. Designed to integrate both a zero-trust security model and hyperscale capabilities, the DE Platform Enclave 802 delivers a secure and scalable processing environment tailored to the individual customer needs. Zero-trust security features include, but are not limited to, rigorous access control, algorithmic fairness, and data isolation. Enclave 802 also supports a machine learning engine for real-time analytics (e.g., machine learning engine 120), auto-scaling features for workload adaptability, and API-based interoperability with third-party services. Security and resource optimization are enhanced by multi-tenancy support, role-based access control, and data encryption both at rest and in transit. The Digital Engineering Platform Enclave 802 may also include one or more of the features described below.

[0067] First, the Digital Engineering Platform Enclave 802 can be designed in accordance with the principles of zero-trust security. In particular, the DE Platform Enclave 802 employs the zero-trust principle to ensure that no implicit trust is assumed between any element, such as the digital model, platform agents, or individual users (e.g., users 104A, 104B) within the system, or their actions. The model is further reinforced by a strict access control mechanism that restricts even the management team (e.g., a team of individuals associated with the platform provider) to predetermined limited access to the Enclave's resources. To further enhance this robust security stance, data encryption is applied both at rest and in transit to effectively mitigate the risk of unauthorized access and data breaches.

[0068] The DE platform enclave 802 can also be designed to maintain isolation and independence. A key aspect of the enclave's architecture is its emphasis on fairness and isolation. Enclave 802 enforces a strong isolation policy by not allowing cryptographic dependencies from external enclaves. The enclave's design also allows for both single-tenant and multi-tenant configurations, further enhancing the isolation of data and processes between customers 806 (e.g., users 104A, 104B). Furthermore, Enclave 802 is designed with a decoupled resource set to minimize interdependencies, thereby increasing system efficiency and autonomy.

[0069] The DE platform Enclave 802 can be designed for scalability and adaptability. The Enclave 802 is engineered to be scalable and adaptable, effectively addressing a variety of operating requirements. For example, the Enclave 802 can incorporate hyperscale-like characteristics in conjunction with zero-trust principles to enable scalable growth and effectively handle high-performance workloads.

[0070] The DE Platform Enclave 802 can be further designed for workflow adaptability, supported by a rigorous access control mechanism. The DE Platform Enclave 802 is designed to accommodate various customer workflows and DE models through its rigorous access control mechanism. This configurability allows for a modular approach to integrate different functions, from data ingestion to algorithm execution, without compromising a zero-trust security posture. The adaptability of Platform 800 gives it high versatility for many use cases while ensuring stable performance and robust security.

[0071] The DE platform enclave 802 can be further designed to enable analysis for robust platform operation. At the heart of the enclave's operational efficiency is a machine learning engine (e.g., machine learning engine 120) capable of performing real-time analysis. This improves decision-making and operational efficiency throughout the platform 800. An auto-scaling mechanism can also be included to enable dynamic resource allocation based on workload demands, further enhancing the platform's responsiveness and efficiency.

[0072] In an exemplary implementation, the DE platform enclave 802 may include several components, as shown in Figure 8 and as described in further detail herein.

[0073] In the embodiment of the DE platform enclave 802 shown in Figure 8, the DE platform enclave 802 includes “monitoring services” and “telemetry services” as part of a “monitoring service cell.” These components focus on maintaining, tracking, and analyzing the performance of platform 800 to ensure optimal service delivery, including advanced machine learning capabilities for real-time analysis.

[0074] In the embodiment of the DE platform enclave 802 shown in Figure 8, the DE platform enclave 802 also includes a "Static Assets Service Cell" that houses the user interface, SDK, command line interface (CLI), and documentation for platform 800.

[0075] In the embodiment of the DE platform enclave 802 shown in Figure 8, the DE platform enclave 802 includes DE platform APIs (e.g., APIs 114, 116) and further includes an "API gateway service cell" that acts as an intermediary for requests between client applications (e.g., digital engineering tools 102, a repository of common V&V products 110, etc.) and platform services.

[0076] In the embodiment of the DE platform enclave 802 shown in Figure 8, the DE platform enclave 802 further includes a “search service cell.” This component facilitates the efficient retrieval of information from the DE platform 800 and enhances the overall functionality of the DE platform 800.

[0077] In the embodiment of the DE platform enclave 802 shown in Figure 8, the DE platform enclave 802 further includes a “logging service cell” and a “control plane service cell.” These components help record and manage operational events and the flow of information within the platform 800.

[0078] As shown in Figure 8, the architecture of the digital engineering platform 800 also includes cloud services 804, which include services that can modify the software for orchestrating the operation of the digital engineering platform, although they cannot interact with customer data. In an exemplary implementation, several cloud resources provide support and infrastructure services to the platform. For example, in the embodiment of the DE platform 800 shown in Figure 8, cloud services 804 include a “Customer IAM Service,” where “IAM” stands for “Identity and Access Management.” The Identity and Access Management Service ensures secure and controlled access to the platform 800.

[0079] In the embodiment of the DE platform 800 shown in Figure 8, the cloud service 804 also includes a “testing service” which includes test tools for verifying the validity of the platform’s operation.

[0080] In the embodiment of the DE platform 800 shown in Figure 8, the cloud service 804 also includes an "orchestration service" for controlling and managing the lifecycle of containers on the platform 800.

[0081] In the embodiment of the DE platform 800 shown in Figure 8, the cloud service 804 also includes an "Artifact service" and a "Version control and build service." These cloud services are crucial for maintaining the progress of projects, code, and instances within the system, while also managing artifacts generated during the product development process.

[0082] As shown in Figure 8, the architecture of the Digital Engineering Platform 800 also includes a Customer Environment 810 with an "Authoritative Source of Truth" 812, customer tools 814, and an optional DE Platform Exclave 816. The Customer Environment 810 is where customer data resides and is processed in a zero-trust manner by the Digital Engineering Platform 800. As previously mentioned, the DE Platform Enclave 802 provides a robust and scalable environment for securely handling critical workloads according to the customer's unique needs, with an emphasis on both zero-trust principles and hyperscale-like characteristics. In some examples, the DE Platform Exclave 816 is located within the Customer Environment 810 to support the customer's digital engineering tasks and operations.

[0083] When a customer 806 (e.g., users 104A, 104B) intends to perform a digital engineering task using a digital engineering platform 800 (e.g., an interconnected digital engineering and authentication ecosystem 100), typical operations include secure data ingestion and controlled data retrieval. Derived data generated through digital engineering operations, such as updated digital model files or revisions of digital model parameters, is stored only within the customer environment 810, and the digital engineering platform 800 may provide tools for accessing metadata of the derived data. An exemplary implementation may include secure data ingestion that leverages zero-trust principles to ensure that customer data is securely uploaded to the customer environment 810 through a pre-validated secure tunnel, such as a Secure Sockets Layer (SSL) tunnel. This may enable direct, secure file transfers to designated cloud storage, such as an S3 bucket, within the customer environment 810. An exemplary implementation may also include controlled data retrieval, where a temporary, pre-authenticated URL generated via a secure token-based mechanism is used for controlled data access, thereby minimizing the risk of malicious interaction. Exemplary embodiments may include immutable derived data, such as transformed data generated through operations like data extraction, which are securely stored within the customer environment 810 while adhering to zero-trust security protocols. Exemplary embodiments may also include a tokenization utility, in which a specialized digital engineering (DE) platform tool called a “tokenizer” is deployed within the customer environment 810 for the secure management of derived metadata in compliance with zero-trust guidelines.

[0084] The customer environment 810 interacts with other elements of the secure digital engineering (DE) platform 800 and includes several features that handle data storage and secure interaction with the platform 800. For example, one element of the customer environment 810 is the “Trusted Authoritative Source” 812, which is the primary repository of customer data, ensuring data integrity and accuracy. Nested within this is the “Customer Bucket,” where data is securely stored with strict access control that restricts access to the data to authorized users or processes through pre-authenticated URL links. This setup ensures uncompromising data security within the customer environment 810 while providing seamless interaction with other elements of the DE platform 800.

[0085] The customer environment 810 also includes additional software tools (e.g., customer tools 814) that may be used based on specific customer requirements. For example, the "DE Tool Host" is a component that handles the data engineering applications necessary for working with customer data. The "DE Tool Host" includes the DET CLI (Data Engineering Tools Command Line Interface), which enables user-friendly command-line operation of DE tools (e.g., digital engineering tools 102). The "DE Platform Agent" ensures smooth communication and management between the customer environment 810 and the elements of the DE Platform 800. Furthermore, there may be another set of optional DE tools designed to support customer-specific data engineering workflows.

[0086] In some cases, an optional feature known as the "DE Platform Exclaver" 816 may be used within the customer environment 810 for enhanced security. The Exclaver 816 operates within the customer's network, overseeing data processing and strictly adhering to zero-trust principles while providing hyperscale-like platform performance. The Exclaver 816 includes a "DE Tool Host" that runs the DE tools and agents necessary for its operation.

[0087] Referring now to Figure 6, an exemplary process 600 for product development is shown. In some implementations, process 600 may be performed by a computing system (e.g., computing system 108) of an interconnected digital engineering and certification ecosystem (e.g., ecosystem 100).

[0088] The operation of process 600 includes receiving design and / or engineering data (D / E data) corresponding to a prototype representation of the product from a user device (602). For example, the user device may correspond to user device 106A or API 106B, and the D / E data may correspond to MBSE files, CAD files, and / or other digital files or information related to the digital prototype, as described above. In some implementations, the product may be a UAV or other type of aircraft, automobile, boat, submersible, industrial robot, spacecraft, satellite, structure, tool, physical device, mobile device, pharmaceutical, chemical product, or biological agent, manufacturing process, or any other complex system (either physical or non-physical) that may be evaluated against a common V&V product.

[0089] The operation of process 600 also includes sending one or more inputs derived from D / E data to one or more digital engineering tools for processing (604). For example, one or more digital engineering tools may correspond to the digital engineering tool 102 described above. In some implementations, at least a subset of one or more digital engineering tools may include model-based systems engineering (MBSE) tools, augmented reality (AR) tools, computer-aided design (CAD) tools, data analysis tools, modeling and simulation (M&S) tools, product lifecycle management (PLM) tools, simulation engines, requirements models, electronics models, test planning models, cost models, scheduling models, software modeling, supply chain models, manufacturing models, cybersecurity models, multi-attribute trade space tools, or mission effect models, or other similar digital engineering tools that may be recognized as engineering design tools by those skilled in the art.

[0090] The operation of process 600 also includes receiving engineering-related data output from one or more digital engineering tools (606). For example, the engineering-related data output may correspond to the results of models, tests, and / or simulations performed by the data engineering tool 102, as described above.

[0091] The operation of process 600 also includes receiving data corresponding to one or more common V&V products associated with the product (608). For example, one or more common V&V products may be digitized regulatory and / or certification standards and may correspond to common V&V products 110A to 110J stored in the repository of common V&V product 110 described above. In some implementations, data corresponding to one or more common V&V products may be received from a user device (for example, by user upload). In some implementations, data corresponding to one or more common V&V products may be received from a regulatory and / or certification authority (for example, via a repository of common V&V products hosted or maintained by the regulatory and / or certification authority).

[0092] The operation of process 600 also includes identifying one or more requirements for a product based on data corresponding to one or more common V&V products (610). For example, one or more requirements may correspond to requirements that must be met in order to certify a product according to a particular common V&V product.

[0093] The operation of process 600 also includes determining whether one or more requirements have been met based on engineering-related data output and data corresponding to one or more common V&V products (612). In some implementations, instead of performing a binary decision, the operation of process 600 may include determining whether one or more requirements are likely to be met by the prototype representation of the product (for example, based on estimated probabilities). In some implementations, determining whether one or more requirements have been met (or are likely to be met) based on engineering-related data output may include determining whether one or more requirements have been met with or without human input.

[0094] The operation of process 600 is to present information on the user device corresponding to engineering-related data output and / or data corresponding to one or more common V&V products, including presenting (614) information that includes indications of whether one or more requirements have been met. In some implementations, the presented information may include indications of the probability that one or more requirements are met by the prototype representation of the product. For example, the information may be presented on the user device in the form of a report, as shown in display 310 of Figure 3 and as described above. In some implementations, the presented information may further include recommended actions that the user of the user device may take to meet one or more requirements. In such implementations, recommended actions may include suggestions to use a specific digital engineering tool from one or more digital engineering tools, suggestions to modify one or more inputs sent to one or more digital engineering tools, suggestions to modify one or more components of the prototype representation of the product, suggestions to replace one or more components of the prototype representation of the product with a previously designed solution, and / or suggestions for a completely or partially new design generated by the system (for example, using the machine learning engine 120).

[0095] The operation of the process includes presenting information on the user device that corresponds to engineering-related data output and / or data corresponding to one or more common V&V products, and then receiving a command from the user device, the command corresponding to one or more user interactions with the user device (616).

[0096] The operation of the process also includes performing one or more operations on the D / E data in response to receiving a command from a user device (618). In some implementations, performing one or more operations on the D / E data may include modifying the D / E data and / or deriving a modified input from the D / E data for transmission to one or more digital engineering tools.

[0097] Additional operations of process 600 may include: In some implementations, process 600 may include storing usage data in a storage device representing received data corresponding to one or more common V&V products, received D / E data, engineering-related data output from one or more digital engineering tools, indications of whether one or more requirements have been met (or are likely to be met), one or more user interactions with user devices, and / or one or more operations on D / E data. Process 600 may also include incorporating applications and services (e.g., application and service 122) that automate or partially automate the determination of whether one or more requirements have been met or partially met. Process 600 may further include incorporating at least a portion of the usage data into a training dataset and training a machine learning model on the training dataset. In some implementations, a machine learning model may be configured to receive as input information about another product designed by another user and to output suggestions for the other user to use one or more digital engineering tools, suggestions to modify one or more inputs sent to one or more digital engineering tools by another user, suggestions to modify one or more components of another prototype representation related to another user, and / or suggestions to replace one or more components of another prototype representation with a previously designed solution. In some implementations, process 600 may also include using stored usage data for one or more sensitivity analyses. In some implementations, process 600 may also include using stored usage data to improve the performance of applications and services (e.g., application and service 122).

[0098] In some implementations, additional actions of process 600 may include checking one or more user credentials before performing one or more operations on the D / E data, and determining, based on the credentials, whether the user is qualified or authorized to perform one or more operations on the D / E data.

[0099] Figure 7 shows examples of computing device 700 and mobile computing device 750 used to implement an implementation of this disclosure. Computing device 700 is intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. Mobile computing device 750 is intended to represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, VR devices, and other similar computing devices. The components, their connections and relationships, and their functions shown herein are intended to be examples only and not limiting. Computing device 700 and / or mobile computing device 750 may form at least part of the user device 106A or API 106B and computing system 108 described above. As stated above, in some implementations, computing system 108 may be a distributed computing system including multiple computing devices such as computing device 700 and / or mobile computing device 750. In other implementations, computing system 108 may include a single computing device. In some implementations, API 106B may be implemented on computing device 700 and / or mobile computing device 750 to relay digital computer files to a non-human artificial user 104B (e.g., an artificial intelligence and / or algorithmic user), the non-human artificial user 104B may itself be implemented on computing device 700 and / or mobile computing device 750 (or on a separate instance of computing device 700 and / or mobile computing device 750).

[0100] The computing device 700 includes a processor 702, memory 704, a storage device 706, a high-speed interface 708, and a low-speed interface 712. In some implementations, the high-speed interface 708 connects to memory 704 and several high-speed expansion ports 710. In some implementations, the low-speed interface 712 connects to the low-speed expansion port 714 and the storage device 706. Each of the processor 702, memory 704, storage device 706, high-speed interface 708, high-speed expansion port 710, and low-speed interface 712 may be interconnected using various buses and mounted on a common motherboard or in other ways as needed. The processor 702 processes instructions for execution within the computing device 700, including instructions stored in memory 704 and / or on the storage device 706, to display graphical information for a graphical user interface (GUI) on an external input / output device such as a display 716 coupled to the high-speed interface 708. In other implementations, multiple processors and / or multiple buses may be used as appropriate, along with multiple memories and multiple types of memory. In addition, multiple computing devices may be connected in such a way that each device performs some of the necessary operations (for example, as a server bank, a group of blade servers, or a multiprocessor system).

[0101] Memory 704 stores information within the computing device 700. In some implementations, memory 704 is one or more volatile memory units. In some implementations, memory 704 is one or more non-volatile memory units. Memory 704 may also be another form of computer-readable medium, such as a magnetic or optical disk.

[0102] The storage device 706 can provide high-capacity storage to the computing device 700. In some implementations, the storage device 706 is or may include computer-readable media such as a floppy disk device, a hard disk device, an optical disk device, a tape device, flash memory, or other similar solid-state memory devices, or an array of devices including devices in a storage area network or other configuration. Instructions can be stored in an information carrier. When the instructions are executed by one or more processing devices such as the processor 702, they perform one or more methods, such as those described above. Instructions can also be stored in one or more storage devices such as computer-readable or machine-readable media, such as memory 704, the storage device 706, or memory on the processor 702.

[0103] The high-speed interface 708 manages bandwidth-intensive operations related to the computing device 700, while the low-speed interface 712 manages less bandwidth-intensive operations. Such function allocations are merely examples. In some implementations, the high-speed interface 708 is coupled to memory 704, to a display 716 (e.g., through a graphics processor or accelerator), and to a high-speed expansion port 710 which may accept various expansion cards. In some implementations, the low-speed interface 712 is coupled to the storage device 706 and the low-speed expansion port 714. The low-speed expansion port 714, which may include various communication ports (e.g., Universal Serial Bus (USB), Bluetooth®, Ethernet, Wireless Ethernet), may be coupled to one or more input / output devices. Such input / output devices may include a scanner 730, a printing device 734, or a keyboard or mouse 736. The input / output devices may also be coupled to the low-speed expansion port 714 through a network adapter 732. Such network input / output devices may include, for example, a switch or a router.

[0104] The computing device 700 may be implemented in many different forms, as shown in Figure 7. For example, the computing device 700 may be implemented as a single standard server 720, or multiple times within a group of such servers. Furthermore, the computing device 700 may be implemented in a personal computer, such as a laptop computer 722. The computing device 700 may also be implemented as part of a rack server system 724, a high-performance computing enclave, or a quantum and / or non-silicon-based computing system. Alternatively, the components of the computing device 700 may be combined with other components of a mobile device, such as a mobile computing device 750. Each of such devices may contain one or more of the computing device 700 and the mobile computing device 750, and the entire system may consist of multiple computing devices communicating with each other.

[0105] The mobile computing device 750 includes, among other components, input / output devices such as a processor 752, memory 764, and display 754, a communication interface 766, and a transceiver 768. The mobile computing device 750 may also include a storage device, such as a microdrive or other device, to provide additional storage. Each of the processor 752, memory 764, display 754, communication interface 766, and transceiver 768 is interconnected using various buses, and some of the components may be mounted on a common motherboard or in other ways as needed. In some implementations, the mobile computing device 750 may include a camera device.

[0106] The processor 752 can execute instructions within the mobile computing device 750, including instructions stored in memory 764. The processor 752 may be implemented as a chipset of a chip containing multiple separate analog and digital processors. For example, the processor 752 may be a composite instruction set computer (CISC) processor, a reduced instruction set computer (RISC) processor, or a minimal instruction set computer (MISC) processor. The processor 752 may also coordinate other components of the mobile computing device 750, such as the user interface (UI), applications run by the mobile computing device 750, and / or control of wireless communications by the mobile computing device 750.

[0107] The processor 752 may communicate with the user through a control interface 758 and a display interface 756 coupled to the display 754. The display 754 may be, for example, a thin-film transistor liquid crystal display (TFT) display, an organic light-emitting diode (OLED) display, or other suitable display technology. The display interface 756 may include appropriate circuitry for driving the display 754 to present graphical and other information to the user. The control interface 758 may receive commands from the user and translate those commands for transmission to the processor 752. In addition, an external interface 762 may provide communication with the processor 752 to enable near-area communication of the mobile computing device 750 with other devices. The external interface 762 may provide, for example, wired communication in some implementations and wireless communication in other implementations, and multiple interfaces may be used.

[0108] Memory 764 stores information within the mobile computing device 750. Memory 764 may be implemented as one or more of the following: one or more computer-readable media, one or more volatile memory units, or one or more non-volatile memory units. Additionally, an expansion memory 774 may be provided and connected to the mobile computing device 750 via an expansion interface 772, which may include, for example, a Single in Line Memory Module (SIMM) card interface. The expansion memory 774 may provide additional storage space to the mobile computing device 750, or it may store applications or other information of the mobile computing device 750. In particular, the expansion memory 774 may contain instructions that execute or supplement the processes described above, and may also contain secure information. Therefore, for example, the expansion memory 774 may be provided as a security module for the mobile computing device 750 and may be programmed with instructions that enable the secure use of the mobile computing device 750. Furthermore, SIMM cards may provide secure applications along with additional information, such as storing identification information on the SIMM card in a way that makes it impossible to hack.

[0109] The memory may include, for example, flash memory and / or non-volatile random-access memory (NVRAM), as considered below. In some implementations, instructions are stored on an information carrier. When an instruction is executed by one or more processing devices, such as processor 752, it performs one or more methods, such as those described above. Instructions may be stored in one or more storage devices, such as one or more computer-readable or machine-readable media, such as memory 764, extended memory 774, or memory on processor 752. In some implementations, instructions may be received as propagated signals, such as via transceiver 768 or external interface 762.

[0110] The mobile computing device 750 may communicate wirelessly through a communication interface 766, which may include digital signal processing circuitry as needed. The communication interface 766 may provide communication under various modes or protocols, including Global System for Mobile communications (GSM) voice calls, Short Message Service (SMS), Enhanced Messaging Service (EMS), Multimedia Messaging Service (MMS) messaging, Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Personal Digital Cellular (PDC), Wideband Code Division Multiple Access (WCDMA®), CDMA2000, and General-Purpose Packet Radio Service (GPRS). Such communication may be conducted, for example, through a transceiver 768 using radio frequencies. Furthermore, short-range communication may be conducted, such as using Bluetooth® or Wi-Fi. In addition, the Global Positioning System (GPS) receiver module 770 may provide the mobile computing device 750 with further navigation and location-related wireless data that may be used as appropriate by applications running on the mobile computing device 750.

[0111] The mobile computing device 750 may also communicate via voice using an audio codec 760 that can receive information spoken by the user and convert that information into usable digital information. Similarly, the audio codec 760 may generate audible audio for the user, for example, through the speaker of the handset of the mobile computing device 750. Such audio may include voice from a voice call, recorded audio (e.g., voice messages, music files, etc.), or audio generated by an application running on the mobile computing device 750.

[0112] The mobile computing device 750 may be implemented in many different forms, as shown in Figure 7. For example, the mobile computing device 750 may be implemented as a telephone device 780, a personal digital assistant 782, and a tablet device (not shown). The mobile computing device 750 may also be implemented as a component of a smartphone, an augmented reality (AR) device, or other similar mobile device.

[0113] Computing devices 700 and / or 750 may also include a USB flash drive. The USB flash drive may store the operating system and other applications. The USB flash drive may include input / output components such as a wireless transmitter or a USB connector that may be inserted into a USB port on another computing device.

[0114] Other embodiments and applications not specifically described herein are also within the scope of the appended claims. Elements of different implementations described herein may be combined to form other embodiments. [Explanation of symbols]

[0115] 100 Interconnected Digital Engineering and Certification Ecosystems 102 Digital Engineering Tools 102A Data Analysis Tool 102B CAD and Finite Element Analysis Tools 102C Simulation Tool 102D~102E Chemical M&S Tools 102F~102G Manufacturing M&S Tools 104 users 104A Human user 104B Artificial User 106A User Device 106B API (or other similar machine-to-machine communication interface) 108 Computing Systems 110 Common V&V Products Regulatory standards related to the development and certification of 110A-110F UAVs 110G Medical Standard 110H Medical Certification Regulations 110I Manufacturing Standard 110J Manufacturing Certification Regulations 112, 112A~112C Digitally Certified Products 114 API / SDK 116 APIs / SDKs 118 Data Storage Units 120 Machine Learning Engines 122 Application and Service Layer 200 Digital Product Development and Certification Workflows 300 Series of Exemplary Displays 302 display 304 display 306 display 308 displays 310 displays 400 flowchart 402 The Physical World 404 Transfer Function Models and Tools 406 The Digital World 408 Digital Product Development 410 Digital Product Testing 412 Digital Product Certification 414 Finalized Digital Product Design 416 Manufacturing 418 Final products in the physical world 500A Monetization Opportunities 500B monetization opportunity 500C Monetization Opportunities 500D Monetization Opportunities 600 processes 700 Computing Devices 702 Processor 704 memory 706 Storage Devices 707 Storage Devices 708 High-Speed ​​Interface 710 High-Speed ​​Expansion Ports 712 Low-speed interface 714 Low-speed expansion ports 716 displays 720 servers 722 Laptop Computers 724 rack server system 730 Scanner 732 Network Adapter 734 Printing Devices 736 Keyboard or Mouse 750 mobile computing devices 752 processors 754 displays 756 Display Interface 758 Control Interface 760 audio codecs 762 External Interface 764 memory 766 Communication Interface 768 Transceiver 770 GPS Receiver Module 772 Expansion Interface 774 Expansion memory 780 Phone Devices 782 Mobile Information Terminal 800 Digital Engineering Platform 802 DE Platform Enclave 804 Cloud Services 806 Customer 810 Customer environment 812 Reliable and authoritative sources 814 Customer Tools 816 DE Platform Exclav

Claims

1. The steps include receiving design and / or engineering data (D / E data) corresponding to the product prototype representation from a user device, The steps include sending one or more inputs derived from the D / E data to one or more digital engineering tools for processing, The steps include receiving engineering-related data output from one or more digital engineering tools, The steps include receiving data corresponding to one or more Common Validation and Verification (V&V) products related to the said product, A step of identifying one or more requirements for the product based on the data corresponding to the one or more common V&V products, A step of determining whether one or more requirements have been met based on the engineering-related data output and the data corresponding to one or more common V&V products, A step of presenting information on the user device corresponding to the engineering-related data output and / or the data corresponding to one or more common V&V products, wherein the presented information includes an indication of whether one or more requirements are met, or an indication of the probability of whether one or more requirements are met by the prototype representation of the product. A step of receiving a command from the user device after presenting the information corresponding to the engineering-related data output and / or the data corresponding to the one or more common V&V products on the user device, wherein the command corresponds to one or more user interactions with the user device. The steps include: performing one or more operations on the D / E data in response to receiving the command from the user device; A method performed by a computer, including the above.

2. The method, performed by a computer according to claim 1, wherein the data corresponding to one or more common V&V products is received from the user device.

3. The computer-based method according to claim 1, wherein the data corresponding to one or more common V&V products is received from a regulatory and / or certification authority.

4. The computer-based method according to claim 1, wherein the product is an aircraft.

5. The computer-based method according to claim 1, wherein at least a subset of the one or more digital engineering tools includes model-based systems engineering (MBSE) tools, augmented reality (AR) tools, computer-aided design (CAD) tools, data analysis tools, modeling and simulation (M&S) tools, product lifecycle management (PLM) tools, simulation engines, requirements models, electronics models, test planning models, cost models, scheduling models, software modeling, supply chain models, manufacturing models, cybersecurity models, multi-attribute trade space tools, or mission effect models.

6. A method performed by a computer according to claim 1, wherein the step of determining whether one or more of the requirements have been met based on the engineering-related data output includes determining whether one or more of the requirements have been met without any human input.

7. The method performed by a computer according to claim 1, wherein the information presented further includes recommended actions that the user of the user device may take to satisfy one or more of the requirements.

8. The computer-based method according to claim 7, wherein the recommended action includes a suggestion to use a specific digital engineering tool of the one or more digital engineering tools, a suggestion to modify the one or more inputs sent to the one or more digital engineering tools, a suggestion to modify one or more components of the prototype representation of the product, and / or a suggestion to replace one or more components of the prototype representation of the product with a previously designed solution.

9. The method performed by a computer according to claim 1, wherein the step of performing the one or more operations on the D / E data includes modifying the D / E data and / or deriving the modified input from the D / E data for transmission to the one or more digital engineering tools.

10. A computer-based method according to claim 1, further comprising the step of storing in a storage device received data corresponding to one or more common V&V products, received D / E data, the engineering-related data output from one or more digital engineering tools, an indication of whether one or more requirements have been met, an indication of the probability that one or more requirements are met by the prototype representation of the product, one or more interactions of the user with the user device, and / or usage data representing one or more operations on the D / E data.

11. The steps include incorporating at least a portion of the aforementioned data used into a training dataset, The steps include training a machine learning model based on the aforementioned training dataset and A computer-based method according to claim 10, further comprising:

12. The aforementioned machine learning model, Receiving information as input about another product designed by another user, Outputting suggestions for the other user to use a specific digital engineering tool of the one or more digital engineering tools, suggestions to modify one or more inputs sent by the other user to the one or more digital engineering tools, suggestions to modify one or more components of another prototype representation related to the other user, suggestions to replace one or more components of the other prototype representation with a previously designed solution, and / or suggestions for a completely or partially new design generated using a machine learning engine. A computer-based method according to claim 11, configured to do the following:

13. The method performed by a computer according to claim 10, further comprising the step of using stored usage data for one or more sensitivity analyses.

14. Before performing one or more operations on the D / E data, the steps include checking one or more user credentials, The steps include determining, based on the one or more credentials, that the user is qualified to perform one or more operations on the D / E data, and A computer-based method according to claim 1, further comprising:

15. The method, performed by a computer according to claim 1, wherein the step of receiving the D / E data from the user device includes receiving a request from a customer environment in the digital engineering platform enclave, the customer environment and the digital engineering platform enclave are managed by different entities, and the request from the customer environment does not modify the production software associated with the digital engineering platform enclave.

16. Memory for storing executable instructions, One or more processing devices coupled to the memory, Receiving design and / or engineering data (D / E data) corresponding to the product prototype representation from the user device. One or more inputs derived from the D / E data are transmitted to one or more digital engineering tools for processing. Receiving engineering-related data output from one or more of the aforementioned digital engineering tools, Receiving data corresponding to one or more Common Validation and Verification (V&V) products related to the said product, Based on the data corresponding to the one or more common V&V products, identify one or more requirements for the product. Based on the aforementioned engineering-related data output and the data corresponding to one or more common V&V products, it is determined whether one or more of the requirements have been met. Presenting on the user device information corresponding to the engineering-related data output and / or the data corresponding to one or more common V&V products, wherein the presented information includes an indication of whether one or more requirements are met, or an indication of the probability of whether one or more requirements are met by the prototype representation of the product. After presenting the information corresponding to the engineering-related data output and / or the data corresponding to the one or more common V&V products on the user device, the user device receives a command, wherein the command corresponds to one or more user interactions with the user device, and In response to receiving the command from the user device, perform one or more operations on the D / E data. One or more processing devices configured to execute the instructions in order to perform an operation including the above, A system that includes this.

17. The system according to claim 16, wherein the data corresponding to one or more common V&V products is received from the user device.

18. The system according to claim 16, wherein the data corresponding to one or more common V&V products is received from regulatory and / or certification authorities.

19. The system according to claim 16, wherein the product is an aircraft.

20. The system according to claim 16, wherein at least a subset of the one or more digital engineering tools includes model-based systems engineering (MBSE) tools, augmented reality (AR) tools, computer-aided design (CAD) tools, data analysis tools, modeling and simulation (M&S) tools, product lifecycle management (PLM) tools, simulation engines, requirements models, electronics models, test planning models, cost models, scheduling models, software modeling, supply chain models, manufacturing models, cybersecurity models, multi-attribute trade space tools, or mission effect models.

21. The system according to claim 16, wherein determining whether one or more of the requirements are met based on the engineering-related data output includes determining whether one or more of the requirements are met without any human input.

22. The system according to claim 16, wherein the information presented further includes recommended actions that the user of the user device may take to satisfy one or more of the requirements.

23. The system according to claim 22, wherein the recommended actions include a suggestion to use a specific digital engineering tool of the one or more digital engineering tools, a suggestion to modify the one or more inputs sent to the one or more digital engineering tools, a suggestion to modify one or more components of the prototype representation of the product, and / or a suggestion to replace one or more components of the prototype representation of the product with a previously designed solution.

24. The system according to claim 16, wherein performing the one or more operations on the D / E data includes modifying the D / E data and / or deriving a modified input from the D / E data for transmission to the one or more digital engineering tools.

25. The system according to claim 16, further comprising storing in a storage device usage data representing the received data corresponding to the one or more common V&V products, received D / E data, the engineering-related data output from the one or more digital engineering tools, the indication of whether one or more requirements have been met, the indication of the probability of whether one or more requirements are met by the prototype representation of the product, the one or more interactions of the user with the user device, and / or the one or more operations performed on the D / E data.

26. The aforementioned operation, This involves incorporating at least a portion of the aforementioned data used into the training dataset, Training a machine learning model based on the aforementioned training dataset and The system according to claim 25, further comprising:

27. The aforementioned machine learning model, Receiving information as input about another product designed by another user, Outputting suggestions for the other user to use a specific digital engineering tool of the one or more digital engineering tools, suggestions to modify one or more inputs sent by the other user to the one or more digital engineering tools, suggestions to modify one or more components of another prototype representation related to the other user, suggestions to replace one or more components of the other prototype representation with a previously designed solution, and / or suggestions for a completely or partially new design generated using a machine learning engine. The system according to claim 26, configured to do the following.

28. The system according to claim 25, further comprising using stored usage data for one or more sensitivity analyses.

29. The aforementioned operation, Before performing the one or more operations on the D / E data, the user's one or more credentials shall be checked. Based on the one or more credentials, it is determined that the user is qualified to perform the one or more operations on the D / E data. The system according to claim 16, further comprising:

30. Receiving design and / or engineering data (D / E data) corresponding to the product prototype representation from the user device. One or more inputs derived from the D / E data are transmitted to one or more digital engineering tools for processing. Receiving engineering-related data output from one or more of the aforementioned digital engineering tools, Receiving data corresponding to one or more Common Validation and Verification (V&V) products related to the said product, Based on the data corresponding to the one or more common V&V products, identify one or more requirements for the product. Based on the aforementioned engineering-related data output and the data corresponding to one or more common V&V products, it is determined whether one or more of the requirements have been met. Presenting on the user device information corresponding to the engineering-related data output and / or the data corresponding to one or more common V&V products, wherein the presented information includes an indication of whether one or more requirements are met, or an indication of the probability of whether one or more requirements are met by the prototype representation of the product. After presenting the information corresponding to the engineering-related data output and / or the data corresponding to the one or more common V&V products on the user device, the user device receives a command, wherein the command corresponds to one or more user interactions with the user device, and In response to receiving the command from the user device, perform one or more operations on the D / E data. One or more non-temporary, machine-readable storage media that store instructions to be executed in order to perform an action including [a specific action].

Images