An interconnected digital engineering and certification ecosystem

Abstract

A computer-implemented method includes receiving design and / or engineering data (D / E data) corresponding to a prototype representation of a product and transmitting one or more inputs derived from the D / E data to one or more digital engineering tools for processing. The method further includes receiving engineering-related data output from the one or more digital engineering tools and receiving data corresponding to one or more common validation and verification (V&V) products. The method further includes identifying one or more requirements for the product based on the data corresponding to the one or more common V&V products, determining whether the one or more requirements are satisfied, and presenting information corresponding to the engineering-related data output and / or the data corresponding to the one or more common V&V products. The method further includes receiving instructions from a user device and performing one or more operations on the D / E data.

Description

[Technical Field]

[0001] Related Applications This application claims priority to U.S. Provisional Patent Application No. 63 / 419,051, filed October 25, 2022, and entitled INTERCONNECTED DIGITAL ENGINEERING AND CERTIFICATION ECOSYSTEM.

[0002] The present disclosure relates to tools for digital engineering (including modeling and simulation applications) and the authentication of digitally engineered products. [Background technology]

[0003] Digital engineering tools, including modeling and simulation tools that accurately visualize physical systems or processes for real-world consideration, enable the agile development of components and / or systems, the validation of which is still largely performed in the physical world using physical manifestations of the digitally engineered components and / or systems (sometimes generally referred to herein as "products"). Summary of the Invention [Means for solving the problem]

[0004] This specification describes an interconnected digital engineering and certification ecosystem that has several advantages over existing technologies for designing, engineering, testing, and certifying products.

[0005] In recent years, digital engineering tools such as modeling and simulation (M&S) tools, computer-aided design (CAD) tools, model-based systems engineering (MBSE) tools, augmented reality (AR) tools, product lifecycle management (PLM) tools, simulation engines, requirements models, electronics models, test planning models, cost models, schedule models, software modeling, supply chain models, manufacturing models, cybersecurity models, multi-attribute trade-space tools, and mission effects models have increased the agility of hardware development and manufacturing by virtualizing physical systems and / or processes for real-world decisions. However, given the current state of these digital engineering tools, several challenges remain.

[0006] First, there are many diverse digital engineering tools (often designed by different parties), which poses interoperability challenges and can lead to vendor lock-in issues. In particular, directly integrating individual digital engineering tools with each other is costly in both time and money, and the number of interfaces between digital engineering tools grows as the square of the number of different digital engineering tools (i.e., N 2(computational complexity). The large number and variety of digital engineering tools that exist can also pose challenges for implementing scalable applications, automation, machine learning, and / or artificial intelligence across digital engineering tools. Greater interoperability between digital engineering tools can play a key role in developing, testing, and certifying products through processes that may include several different digital engineering tools used in parallel or sequentially. Therefore, seamless interoperability between digital engineering tools can be desirable for implementing such processes by enabling the development of a "digital thread" or pipeline that stitches together the inputs and outputs of multiple digital engineering tools for a specific task.

[0007] Second, due to the highly technical nature of many digital engineering tools, effectively operating such tools often requires highly specialized skill sets, which limits the number of individuals qualified to use these digital engineering tools. Furthermore, an individual skilled in using one digital engineering tool (e.g., a CAD tool produced by a first software company) may not be qualified to use a different type of digital engineering tool (e.g., an MBSE tool), or even a similar digital engineering tool produced by a different company (e.g., a CAD tool produced by a second software company). This applies not only to use of tools via their custom graphical user interfaces, but also via their tool-specific or vendor-specific APIs, which may also require highly specialized skill sets.

[0008] Third, not only may products and solutions designed using one digital engineering tool not be shareable among other digital engineering tools (e.g., due to a lack of interoperability), but in some cases, previously designed products and solutions may not be shareable with or searchable by others using the same digital engineering tools to solve similar problems. For example, there may not be a repository of previously designed products, solutions, etc. for sharing information about such products, solutions, etc. among individuals within the same team, company, technical field, etc. Furthermore, even if such a repository of previously designed products and solutions does exist, it is unlikely to contain information about how and why previously designed products and solutions were arrived at or an easy way to reuse previous engineering work from models, which could potentially be used to limit duplication of effort and / or provide useful insights for individuals working on similar, but slightly different, products or problems. This can result in many engineering problems needing to be redeveloped from scratch rather than building on the results of past efforts.

[0009] Fourth, products and solutions designed using digital engineering often require the use of many different tools that not everyone knows how to use. For example, a model may be built using a specific MBSE tool, and someone who needs access to the model (or the data generated from the model) may not know how to use this tool. This problem, combined with the fact that many complex systems use many different types of tools, means that to understand such a system, individuals may need to know how to use many different tools, which may be extremely rare. This problem is further exacerbated by the fact that those reviewing information for product certification may not be familiar with some or all of the digital engineering tools and may attempt to review all of the data in legacy formats (e.g., PDF reports). This lack of usability between different modeling tools can cause significant delays and increased costs when developing new products, especially if different people or organizations have different technical skill sets, as models cannot be easily shared between them.

[0010] For the reasons stated above, most digital engineering tools today are still built by humans for humans in a world increasingly driven by machine-to-machine autonomy. For example, when designing a complex system like an aircraft, various regulatory standards must be adhered to, which may require many different models and simulations (and, consequently, the use of many different digital engineering tools) to evaluate. Today, such efforts require the collaboration of numerous highly specialized subject matter experts who consult many regulatory standards documents, which inevitably involves many slow and costly human steps in the design and engineering process. Furthermore, current certification processes typically require manufacturing physical manifestations of digitally engineered components and / or systems for evaluation in the physical world (e.g., for physical testing), which can slow down the iterative design and engineering process.

[0011] The interconnected digital engineering and authentication ecosystem (sometimes referred to as the “digital engineering metaverse”) described herein addresses each of these problems and more. Among other things, the interconnected digital engineering and authentication ecosystem can include computing systems (e.g., including networked centralized or distributed computing subsystems or components) that interface (e.g., via application programming interfaces (APIs) and / or software development kits (SDKs)) with various centralized or distributed digital engineering tools, which can be separate from the computing system or can themselves be considered part of the computing system. The digital engineering tools can be interfaced by APIs, and / or the SDK may enable users of the ecosystem (including providers of the digital engineering tools) to develop their own APIs for their tools or models to enable their tools or models to interact with the system. For example, a new company could create a new MBSE tool and then use the SDK to add its tool to the ecosystem, thereby enabling its tool to automatically interoperate with other tools in the ecosystem via APIs. The new company could then potentially maintain that API over time, preventing administrators across the ecosystem from having to maintain all of the different APIs for all of the different tools. This architecture can have the benefit of increasing the ease of interoperability between digital engineering tools.For example, rather than requiring that each individual digital engineering tool be integrated with every other individual digital engineering tool in the ecosystem, a computing system may enable interoperable use of multiple digital engineering tools implemented on multiple other computing systems (or, in some cases, within the same computing system), so long as each of the tools is integrated with the computing system. Further, rather than requiring a user of the digital engineering tools to separately interact with various digital engineering tools to perform modeling and simulation, a computing system may enable the user to interact with and utilize a single user interface of a computing system of the ecosystem, which in turn interfaces with many digital engineering tools. This may result in a gentler learning curve for users who only need to become familiar with a single user interface (e.g., a user interface associated with the computing system), rather than several different user interfaces (e.g., associated with various digital engineering tools). This may also reduce the number of interfaces between digital engineering tools to N. 2 This can be simplified from computational complexity of ∑ to N, where N represents the number of digital engineering tools included in the ecosystem, which in turn can easily lead to scalable applications, automation, and / or machine learning and artificial intelligence across various digital engineering tools.

[0012] An interconnected digital engineering and certification ecosystem also has the advantage of including digitized regulatory and certification standards, compliance, calculations, and tests (e.g., for product and / or solution development, testing, and certification), which may enable users to incorporate relevant regulatory and certification standards, compliance, calculations, and test data directly into their digital engineering workflows. Regulatory and certification standards, compliance, calculations, and tests may be referred to herein as “common validation and verification (V&V) products.” In some implementations, computing systems in the ecosystem can interface with regulatory and / or certification authorities (e.g., via websites operated by the authorities) to retrieve digitized common V&V products published by the authorities that may be important to the product the user is designing. In some implementations, users can upload digitized common V&V products to the ecosystem themselves. Including digitized common V&V products in the ecosystem may be particularly beneficial for the completion of complex systems engineering projects, where many regulatory requirements may need to be met using several different digital engineering tools. By linking both digital engineering tools and a common digitized V&V product, the entire product design and engineering process (or parts of it) can be digitized, eliminating or reducing time-consuming and costly steps (e.g., human review of regulatory standards to identify regulatory requirements, human determination of which digital engineering tools are needed, and human evaluation of whether the regulatory requirements are met).For example, a computing system in a digital engineering and certification ecosystem may be configured to process regulatory and / or certification data corresponding to a digitized common V&V product and engineering-related data output received from one or more digital engineering tools to automatically evaluate whether one or more regulatory and / or certification requirements specified in the common V&V product are met. The computing system may generate a report, which may be presented to a user in an easy-to-read format and may even include recommendations for improvements to the user's product's digital prototype (e.g., to satisfy unsatisfied regulatory and / or certification requirements). Importantly, all of this can be done without any physical manifestation of the manufactured product and without physical testing. As digital models and simulations continue to become increasingly high-fidelity, certification of products such as unmanned aerial vehicles or other aircraft may also be performed digitally, saving the time, cost, and materials associated with physical evaluation and certification of products. Although unmanned aerial vehicles and other aerial vehicles are referenced throughout this specification as exemplary products, the ecosystem could be readily used to design, engineer, test, and / or certify any product or solution (e.g., automobiles, pharmaceuticals, medical devices, processes, etc.) that can be developed using digital engineering tools and / or is subject to regulatory and / or certification requirements.

[0013] An interconnected digital engineering and certification ecosystem also has the advantage of providing a single computing system (which may be a centralized or distributed computing system) through which various types of data flow throughout the design, engineering, testing, and / or certification process. For example, data about prototypes, common V&V products, the use of digital engineering tools to meet specific common V&V products, the success or failure of specific models and simulations, and various design iterations of a product can all be configured to flow securely through and corroborated by the ecosystem's computing systems (e.g., using zero trust security). In some implementations, these data can be tracked and stored. This stored data can be audited for various purposes (e.g., to prevent security breaches or to perform data quality control). The stored data can also be searched to identify patterns within the data (e.g., using a machine learning engine). For example, after many uses of the digital engineering and certification ecosystem by subject matter experts, patterns in the stored data may be used to determine which digital engineering tools are most useful for meeting particular regulatory requirements, to suggest adjustments to inputs or parameters to effectively run models and simulations, to perform sensitivity analyses on particular designs, to design or partially design systems using machine learning and artificial intelligence, etc. This may have the advantage of making the digital engineering and certification ecosystem increasingly user-friendly for subject matter non-experts, who may be assisted by computing systems throughout the design and engineering process based on data collected from more specialized and / or experienced users, accelerating the overall engineering and certification process.

[0014] An interconnected digital engineering and certification ecosystem may further have the advantage of enabling the development of a repository of previous designs and / or solutions already evaluated in connection with one or more common V&V products that can be easily reused with minimal additional engineering effort. Such designs and / or solutions can be suggested to users (e.g., both human users and artificial intelligence users) for use as is or as starting points for modifications, thereby reducing duplicated effort and streamlining the design, engineering, testing, and certification process. In some implementations, the repository may be searchable by users to identify previous designs and / or solutions generated by others. In some implementations, the repository (or specific elements within the repository) may be specific to users with specific credentials (e.g., users associated with a particular company, team, technical field, etc.) to avoid disclosure of confidential material while still facilitating effective collaboration. In some cases, user credentials may additionally or alternatively be used in an interconnected digital engineering and certification ecosystem for other purposes, such as tailoring the types of digital engineering tools (or functions within digital engineering tools) that users may access. For example, user credentials may correspond to the user's skill level and may be checked to ensure that the user is not overwhelmed by features of the digital engineering tool that may be beyond the user's skill set to use effectively.

[0015] An interconnected digital engineering and certification ecosystem can have the additional benefit of allowing highly valuable digital engineering models to be shared while still protecting the intellectual property contained within the models. Many modern technology development projects involve multiple parties (e.g., a customer, a prime integrator, suppliers, etc.) working together and needing access to each other's models, but with different access permissions to the data. This system allows for detailed specification of exactly which data within a model should be shared with each individual party, without exposing all of the data to the parties. This selective sharing of information allows for measurement and tracking of which data is consumed by each party (e.g., sharing only the inputs and outputs of a hydrodynamic pressure model) and how much data is consumed (e.g., how many runs of a hydrodynamic model have been performed). This measurement and tracking enables new business models based on the creation of models and data that can be monitored and monetized. In some implementations, this measurement and tracking may extend beyond the initial sharing of data to measure and / or track subsequent or derivative uses of the data by third parties not involved in the initial sharing agreement. For example, a prime contractor may share data with a first government agency, which may then freely share data with a second government agency, with the prime contractor having the ability to allow / disallow, track, and potentially monetize this further sharing. Such an implementation may have the advantage of allowing for extremely in-depth capture of model data.

[0016] Other features and advantages of the present invention will become apparent from the following description and from the claims. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0017] A general aspect of the present disclosure may include a computer-implemented method that includes receiving design and / or engineering data (D / E data) corresponding to a prototype representation of a product from a user device. The method also includes transmitting one or more inputs derived from the D / E data to one or more digital engineering tools for processing. The method also includes receiving engineering-related data output from the one or more digital engineering tools. The method also includes receiving data corresponding to one or more common validation and verification (V&V) products associated with the product. The method also includes identifying one or more requirements for the product based on the data corresponding to the one or more common V&V products. The method also includes determining whether the one or more requirements have been satisfied based on the engineering-related data output and the data corresponding to the one or more common V&V products. The method also includes presenting, at the user device, information corresponding to the engineering-related data output and / or the data corresponding to the one or more common V&V products. The presented information may include an indication of whether the one or more requirements have been satisfied or an indication of the probability that the one or more requirements are satisfied by the prototype representation of the product. The method also includes receiving instructions from the user device after presenting, at the user device, information corresponding to the engineering-related data output and / or data corresponding to the one or more common V&V products. In some implementations, the instructions correspond to one or more user interactions with the user device. The method also includes performing one or more operations on the D / E data in response to receiving the instructions from the user device.

[0018] In some implementations, data corresponding to one or more common V&V products may be received from a user device. In some implementations, data corresponding to one or more common V&V products may be received from a regulatory and / or certification authority. In some implementations, the product may be an aircraft. In some implementations, at least a subset of the one or more digital engineering tools may include a model-based systems engineering (MBSE) tool, an augmented reality (AR) tool, a computer-aided design (CAD) tool, a data analysis tool, a modeling and simulation (M&S) tool, a product lifecycle management (PLM) tool, a simulation engine, a requirements model, an electronics model, a test planning model, a cost model, a schedule model, a software modeling, a supply chain model, a manufacturing model, a cybersecurity model, a multi-attribute tradespace tool, or a mission effects model. In some implementations, determining whether the one or more requirements are satisfied based on the engineering-related data output may include determining whether the one or more requirements are satisfied without any human input. In some implementations, the presented information may include recommended actions that a user of the user device can take to satisfy the one or more requirements. In some implementations, the recommended actions may include a suggestion to use a particular one of the one or more digital engineering tools, a suggestion to modify one or more inputs sent to the one or more digital engineering tools, a suggestion to modify one or more components of the prototype representation of the product, and / or a suggestion to replace one or more components of the prototype representation of the product with a previously designed solution. In some implementations, performing one or more operations on the D / E data may include modifying the D / E data and / or deriving modified inputs from the D / E data for sending to the one or more digital engineering tools.In some implementations, the computer-implemented method may include storing, in a storage device, usage data representing received data corresponding to one or more common V&V products, received D / E data, engineering-related data output from one or more digital engineering tools, an indication of whether one or more requirements have been met, an indication of a probability that the one or more requirements are met by the prototype representation of the product, one or more interactions of a user with a user device, and / or one or more manipulations of the D / E data. In some implementations, the computer-implemented method may include incorporating at least a portion of the usage data into a training dataset. The implementation may also include training a machine learning model based on the training dataset. In some implementations, the machine learning model may be configured to receive as input information about another product being designed by another user and output: a suggestion for the other user to use a particular one of the one or more digital engineering tools; a suggestion to modify one or more inputs submitted by the other user to the one or more digital engineering tools; a suggestion to modify one or more components of another prototype representation associated with the other user; a suggestion to replace one or more components of the other prototype representation with a previously designed solution; and / or a suggestion of a completely or partially new design generated using the machine learning engine. In some implementations, the computer-implemented method may include using the stored usage data for one or more sensitivity analyses. In some implementations, the computer-implemented method may include checking one or more credentials of the user before performing one or more operations on the D / E data. Implementations may also include determining that the user may be qualified to perform one or more operations on the D / E data based on the one or more credentials.

[0019] Another general aspect of the present disclosure may include a system including a memory for storing instructions, which may be executable, and one or more processing devices coupled to the memory, the one or more processing devices configured to execute the instructions to perform operations. The operations include receiving design and / or engineering data (D / E data) corresponding to a prototype representation of a product from a user device. The operations also include transmitting one or more inputs derived from the D / E data to one or more digital engineering tools for processing. The operations also include receiving engineering-related data output from the one or more digital engineering tools. The operations also include receiving data corresponding to one or more common validation and verification (V&V) products associated with the product. The operations also include identifying one or more requirements for the product based on the data corresponding to the one or more common V&V products. The operations also include determining whether the one or more requirements have been satisfied based on the engineering-related data output and the data corresponding to the one or more common V&V products. The operations also include presenting information corresponding to the engineering-related data output and / or the data corresponding to the one or more common V&V products at the user device. The presented information may include an indication of whether one or more requirements have been met, or an indication of the probability that the one or more requirements are met by the prototype representation of the product. In some implementations, the operations also include receiving instructions from the user device after presenting, at the user device, information corresponding to the engineering-related data output and / or data corresponding to the one or more common V&V products, the instructions corresponding to one or more user interactions with the user device. The operations also include performing one or more operations on the D / E data in response to receiving the instructions from the user device.

[0020] In some implementations, data corresponding to one or more common V&V products may be received from a user device. In some implementations, data corresponding to one or more common V&V products may be received from a regulatory and / or certification authority. In some implementations, the product may be an aircraft. In some implementations, at least a subset of the one or more digital engineering tools may include a model-based systems engineering (MBSE) tool, an augmented reality (AR) tool, a computer-aided design (CAD) tool, a data analysis tool, a modeling and simulation (M&S) tool, a product lifecycle management (PLM) tool, a simulation engine, a requirements model, an electronics model, a test planning model, a cost model, a schedule model, a software modeling, a supply chain model, a manufacturing model, a cybersecurity model, a multi-attribute tradespace tool, or a mission effects model. In some implementations, determining whether the one or more requirements are satisfied based on the engineering-related data output may include determining whether the one or more requirements are satisfied without any human input. In some implementations, the presented information may include recommended actions that a user of the user device can take to satisfy the one or more requirements. In some implementations, the recommended actions may include a suggestion to use a particular one of the one or more digital engineering tools, a suggestion to modify one or more inputs sent to the one or more digital engineering tools, a suggestion to modify one or more components of the product prototype representation, and / or a suggestion to replace one or more components of the product prototype representation with a previously designed solution. In some implementations, performing one or more operations on the D / E data may include modifying the D / E data and / or deriving modified inputs from the D / E data for sending to the one or more digital engineering tools.In some implementations, the operations may include storing, in a storage device, received data corresponding to one or more common V&V products, received D / E data, engineering-related data output from one or more digital engineering tools, an indication of whether one or more requirements have been met, an indication of the probability that the one or more requirements are met by the prototype representation of the product, one or more interactions of the user with the user device, and / or usage data representing one or more manipulations of the D / E data. In some implementations, the operations may include incorporating at least a portion of the usage data into a training dataset. In some implementations, the operations may also include training a machine learning model based on the training dataset. In some implementations, the machine learning model may be configured to receive as input information about another product being designed by another user, and output a suggestion that the other user use a particular one of the one or more digital engineering tools, a suggestion to modify one or more inputs submitted to the one or more digital engineering tools by the other user, a suggestion to modify one or more components of another prototype representation associated with the other user, a suggestion to replace one or more components of the other prototype representation with a previously designed solution, and / or a suggestion of a completely or partially new design generated using the machine learning engine. In some implementations, the operations may include using the stored usage data for one or more sensitivity analyses. In some implementations, the operations may include checking one or more credentials of the user before performing one or more operations on the D / E data. In some implementations, the operations may also include determining that the user may be eligible to perform one or more operations on the D / E data based on the one or more credentials.

[0021] Another general aspect of the present disclosure may include one or more non-transitory machine-readable storage media storing instructions executed to perform operations. The operations include receiving design and / or engineering data (D / E data) corresponding to a prototype representation of a product from a user device. The operations also include sending one or more inputs derived from the D / E data to one or more digital engineering tools for processing. The operations also include receiving engineering-related data output from the one or more digital engineering tools. The operations also include receiving data corresponding to one or more common validation and verification (V&V) products associated with the product. The operations also include identifying one or more requirements for the product based on the data corresponding to the one or more common V&V products. The operations also include determining whether the one or more requirements have been satisfied based on the engineering-related data output and the data corresponding to the one or more common V&V products. The operations also include presenting information corresponding to the engineering-related data output and / or the data corresponding to the one or more common V&V products at the user device. The presented information may include an indication of whether one or more requirements have been met, or an indication of the probability that the one or more requirements are met by the prototype representation of the product. In some implementations, the operations also include receiving instructions from the user device after presenting, at the user device, information corresponding to the engineering-related data output and / or data corresponding to the one or more common V&V products, the instructions corresponding to one or more user interactions with the user device. The operations also include performing one or more operations on the D / E data in response to receiving the instructions from the user device.

[0022] In some implementations, data corresponding to one or more common V&V products may be received from a user device. In some implementations, data corresponding to one or more common V&V products may be received from a regulatory and / or certification authority. In some implementations, the product may be an aircraft. In some implementations, at least a subset of the one or more digital engineering tools may include a model-based systems engineering (MBSE) tool, an augmented reality (AR) tool, a computer-aided design (CAD) tool, a data analysis tool, a modeling and simulation (M&S) tool, a product lifecycle management (PLM) tool, a simulation engine, a requirements model, an electronics model, a test planning model, a cost model, a schedule model, a software modeling, a supply chain model, a manufacturing model, a cybersecurity model, a multi-attribute tradespace tool, or a mission effects model. In some implementations, determining whether one or more requirements have been met based on the engineering-related data output may include determining whether the one or more requirements have been met without any human input. In some implementations, the presented information may include recommended actions that a user of the user device can take to meet the one or more requirements. In some implementations, the recommended actions may include a suggestion to use a particular one of the one or more digital engineering tools, a suggestion to modify one or more inputs sent to the one or more digital engineering tools, a suggestion to modify one or more components of the product prototype representation, and / or a suggestion to replace one or more components of the product prototype representation with a previously designed solution. In some implementations, performing one or more operations on the D / E data may include modifying the D / E data and / or deriving modified inputs from the D / E data for sending to the one or more digital engineering tools.In some implementations, the operations may include storing, in a storage device, received data corresponding to one or more common V&V products, received D / E data, engineering-related data output from one or more digital engineering tools, an indication of whether one or more requirements have been met, an indication of the probability that the one or more requirements are met by the prototype representation of the product, one or more interactions of the user with the user device, and / or usage data representing one or more manipulations of the D / E data. In some implementations, the operations may include incorporating at least a portion of the usage data into a training dataset. In some implementations, the operations may also include training a machine learning model based on the training dataset. In some implementations, the machine learning model may be configured to receive as input information about another product being designed by another user, and output a suggestion that the other user use a particular one of the one or more digital engineering tools, a suggestion to modify one or more inputs submitted to the one or more digital engineering tools by the other user, a suggestion to modify one or more components of another prototype representation associated with the other user, a suggestion to replace one or more components of the other prototype representation with a previously designed solution, and / or a suggestion of a completely or partially new design generated using the machine learning engine. In some implementations, the operations may include using the stored usage data for one or more sensitivity analyses. In some implementations, the operations may include checking one or more credentials of the user before performing one or more operations on the D / E data. In some implementations, the operations may also include determining that the user may be eligible to perform one or more operations on the D / E data based on the one or more credentials. [Brief explanation of the drawings]

[0023] [Figure 1]FIG. 1 illustrates an exemplary interconnected digital engineering and authentication ecosystem and a digitally authenticated product. [Figure 2A] 1 is a flow diagram illustrating an example workflow involving an interconnected digital engineering and certification ecosystem. [Figure 2B] 1 is a flow diagram illustrating an example workflow involving an interconnected digital engineering and certification ecosystem. [Figure 3] 2A-2B show a series of exemplary displays shown on a user device corresponding to the exemplary workflow of FIGS. 2A-2B. [Figure 4] 1 is a flow diagram illustrating an exemplary product design process using an interconnected digital engineering and certification ecosystem. [Figure 5] FIG. 1 illustrates how an interconnected digital engineering and certification ecosystem can be monetized. [Figure 6] 1 is a flow diagram illustrating a process for product development performed by computing systems of an interconnected digital engineering and authentication ecosystem. [Figure 7] FIG. 1 illustrates an example computing environment. [Figure 8] FIG. 1 illustrates an exemplary architecture of an interconnected digital engineering and certification ecosystem. DETAILED DESCRIPTION OF THE INVENTION

[0024] This disclosure describes an interconnected digital engineering and certification ecosystem that can enable new capabilities and improve processes for digital product development, including digital design, digital engineering, digital testing, and digital certification of products. For purposes of this disclosure, the terms "design" and "engineer" are used interchangeably and are broadly defined to encompass the process of intelligently developing a product to solve a specific problem (e.g., to improve performance, enhance aesthetic appeal, meet one or more regulatory requirements, etc.).

[0025] 1 illustrates an exemplary interconnected digital engineering and authentication ecosystem 100 and examples of digitally authenticated products 112A-112C (collectively referred to as digitally authenticated products 112). For example, in some implementations, digitally authenticated product 112A can be an unmanned aerial vehicle (UAV) or other aircraft, digitally authenticated product 112B can be a drug or other chemical or biological compound, and digitally authenticated product 112C can be a process, such as a manufacturing process. Generally, digitally authenticated product 112 can include any product, process, or solution that can be developed, tested, or authenticated (in part or in whole) using digital engineering tools (e.g., digital engineering tools 102). In some implementations, digitally authenticated product 112 may not be limited to physical products and can also include non-physical products (e.g., processes, software, etc.). While physical and physically interacting systems often require multiple digital engineering tools to assess compliance with a common V&V product simply out of M&S necessity, many complex non-physical systems may also require multiple digital engineering tools for product development, testing, and / or certification. With this in mind, various other possibilities for digitally certified products will be recognized by those skilled in the art.

[0026] Digitally authenticated products 112 can be designed and / or authenticated using an interconnected digital engineering and authentication ecosystem 100. The interconnected digital engineering and authentication ecosystem 100 can include user devices 106A or APIs (or other similar machine-to-machine communication interfaces) 106B operated by users (e.g., human users 104A of various skill levels, or artificial users 104B, such as algorithms, artificial intelligence, or other software), and a computing system 108 connected to (and / or including) a data storage unit 118, a machine learning engine 120, and an application and services layer 122. For clarity, all users selected from the various potential human users 104A or artificial users 104B are referred to herein simply as users 104. In some implementations, the computing system 108 can be a centralized computing system, while in other implementations, the computing system 108 can be a distributed computing system. In some cases, the user 104 may be considered part of the ecosystem 100 , while in other embodiments, the user 104 may be considered separate from the ecosystem 100 .The ecosystem 100 includes one or more digital engineering tools 102 (e.g., data analysis tools 102A, CAD and finite element analysis tools 102B, simulation tools 102C, pharmaceutical M&S tools 102D-102E, manufacturing M&S tools 102F-102G, etc.) and common V&V products 110 (e.g., regulatory standards 110A-110F related to the development and certification of UAVs, medical standards 110G (e.g., CE marking (Europe), FCC Declaration of Conformity (USA), IECEE CB Scheme (Europe, North America, parts of Asia & Australia), CDSCO (India), FDA (USA), etc.), medical certification regulations 110H (e.g., ISO 13485, ISO 14971, ISO 9001, ISO 62304, ISO 10993, ISO 15223, ISO 11135, ISO 11137, ISO 11138, etc.), and / or 11607, IEC 60601, etc.), manufacturing standards 110I (e.g., ISO 9001, ISO 9013, ISO 10204, EN 1090, ISO 14004, etc.), and manufacturing certification regulations 110J (e.g., General Certification of Conformity (GCC), etc.).

[0027] The computing system 108 of the ecosystem 100 is centrally located within the architecture of the ecosystem 100 and is configured to communicate with (e.g., receive data from) and send data to (e.g., receive data from) the user devices 106A or APIs 106B (e.g., APIs associated with the artificial users 104B), the digital engineering tools 102 (e.g., via application programming interfaces [APIs] / software development kits [SDKs] 114), and the repository of common V&V products 110 (e.g., via APIs / SDKs 116). For example, the computing system 108 may be configured to communicate with the user devices 106A and / or APIs 106B to send or receive data corresponding to design prototypes, information about the user (e.g., user credentials), engineering-related input / output associated with the digital engineering tools 102, digitized common V&V products, evaluations of product designs, user instructions (e.g., search requests, data processing instructions, etc.), and so forth. The computing system 108 may also be configured to communicate with one or more digital engineering tools 102 to send engineering-related inputs for performing analyses, models, simulations, tests, etc., and to receive engineering-related outputs related to the results. The computing system 108 may also be configured to communicate with a repository of common V&V products 110 to retrieve data corresponding to one or more digitized common V&V products 110 and / or upload new common V&V products (e.g., new common V&V products received from users 104) to the repository of common V&V products 110. All communications may be transmitted and secured securely using, for example, methods that rely on zero trust security.

[0028] The computing system 108 can process and / or store the data it receives, and in some implementations (e.g., using storage 118), can access a machine learning engine 120 and / or an application and services layer 122 (either included as part of the computing system 108 or external to the computing system 108) to identify useful insights based on the data, as described further herein. The centralized location of the computing systems 108 within the architecture of the ecosystem 100 has many advantages, including reducing the technical complexity of integrating various digital engineering tools 102, enhancing the product development experience of the users 104, intelligently connecting common V&V products (e.g., standards 110A-110F) to the digital engineering tools 102 that are most useful for meeting requirements related to the common V&V products, and enabling monitoring, storage, and analysis of various data flowing between elements of the ecosystem 100 throughout the product development process. In some implementations, data flowing through (and potentially stored by) computing system 108 may also be auditable to prevent security breaches, perform data quality control, and the like.

[0029] 1 , a user 104 can use the digital engineering and certification ecosystem 100 to manufacture a digitally certified UAV 112B. For example, the user 104 may be primarily interested in certifying the UAV as meeting the requirements of a particular regulatory standard 110E (e.g., “MIL-HDBK 516C 4.1.4—Failure Condition”) regarding the failure condition of the UAV. In this usage scenario, the user 104 can develop a digital prototype of the UAV on the user device 106A or using the API 106B and can transmit the prototype data (e.g., as at least one of a CAD file, an MBSE file, etc.) to the computing system 108. Along with the prototype data, the user 104 may transmit additional data via the user device 106A, including an indication of a common V&V product (e.g., regulatory standard 110E) for which the user 104 is interested in certifying the product, user credential information for accessing one or more capabilities of the computing system 108, and / or instructions for running one or more digital models, tests, and / or simulations using a subset of the digital engineering tools 102.

[0030] 1 , a user 104 can use the digital engineering and certification ecosystem 100 to produce a digitally certified drug, chemical compound, or biological agent 112A. For example, the user 104 may be primarily interested in certifying the drug, chemical compound, or biological agent 112A as meeting the requirements of a particular medical standard 110G and medical certification regulation 110H. In this usage scenario, the user 104 can develop a digital prototype of the drug, chemical compound, or biological agent on the user device 106A or using the API 106B and transmit the prototype data (e.g., as a molecular modeling file) to the computing system 108. Along with the prototype data, the user 104 may transmit additional data via the user device 106A, including an indication of the common V&V products for which the user 104 is interested in certifying the product (e.g., Medical Standard 110G and Medical Certification Regulation 110H), user credential information for accessing one or more capabilities of the computing system 108, and / or instructions for running one or more digital models, tests, and / or simulations using a subset of the digital engineering tools 102 (e.g., pharmaceutical M&S tools 102D-102E).

[0031] 1 , a user 104 can use the digital engineering and certification ecosystem 100 to produce a digitally certified manufacturing process 112C. For example, the user 104 may be primarily interested in certifying the manufacturing process 112C as meeting the requirements of a particular manufacturing standard 110I and manufacturing certification regulation 110J. In this usage scenario, the user 104 can develop a digital prototype of the manufacturing process on a user device 106A or using an API 106B and can transmit the prototype data to a computing system 108. Along with the prototype data, the user 104 may transmit additional data via the user device 106A, including an indication of common V&V products (e.g., manufacturing standards 110I and manufacturing certification regulations 110J) for which the user 104 is interested in certifying the process, user credential information for accessing one or more capabilities of the computing system 108, and / or instructions for running one or more digital models, tests, and / or simulations using a subset of the digital engineering tools 102 (e.g., manufacturing M&S tools 102F-102G).

[0032] In any of the above examples, the computing system 108 can receive data transmitted from the user device 106A and / or the API 106B and can process the data to evaluate whether common V&V products of interest (e.g., regulatory standards 110E, medical standards 110G, medical certification regulations 110H, manufacturing standards 110I, manufacturing certification regulations 110J, etc.) are satisfied by the user's digital prototype. For example, this can include communicating with a repository of common V&V products 110 (via the API / SDK 116) to retrieve relevant common V&V products of interest and processing regulatory and / or certification data associated with the common V&V products to identify one or more requirements for a UAV prototype, a drug, chemical compound, or biological agent prototype, a manufacturing process prototype, etc. In some implementations, the repository of common V&V products 110 can be hosted by a regulatory and / or certification authority (or another third party), and retrieving the regulatory and / or certification data can include interfacing with one or more data resources maintained by the regulatory and / or certification authority (or another third party) using an API / SDK 116. In some implementations, the regulatory and / or certification data can be provided directly by a user 104 via a user device 106A and / or API 106B (e.g., along with prototype data).

[0033] Evaluating whether common V&V products of interest (e.g., regulatory standards 110E, medical standards 110G, medical certification regulations 110H, manufacturing standards 110I, manufacturing certification regulations 110J, etc.) are met by the user's digital prototype may also include processing prototype data received from user device 106A or API 106B to determine whether one or more identified requirements are indeed met. In some implementations, computing system 108 may include one or more plug-ins, local applications, etc. to process prototype data directly at computing system 108. In some implementations, the computing system may simply preprocess received prototype data (e.g., to derive inputs for digital engineering tool 102) and then send instructions and / or input data to a subset of digital engineering tool 102 via API / SDK 114 for further processing.

[0034] Not all digital engineering tools 102 are necessarily required to meet a particular regulatory and / or certification standard. Thus, in the UAV example given in FIG. 1 , the computing system 108 may determine that only data analysis tools 102A and finite element analysis tools 102B are required to meet regulatory standard 110E regarding failure conditions. In the drug, chemical compound, or biological agent example given in FIG. 1 , the computing system 108 may determine that only drug M&S tools 102D-102E are required to meet medical standard 110G and medical certification regulation 110H. In the manufacturing process example given in FIG. 1 , the computing system 108 may determine that only manufacturing M&S tools 102F-102G are required to meet manufacturing standard 110I and manufacturing certification regulation 110J. In other implementations, the user 104 may self-identify a specific subset of digital engineering tools 102B to be used to meet a common V&V product of interest, provided that the user 104 is a qualified subject matter expert. In other implementations, the user 104 may input several proposed digital engineering tools 102 to the computing system 108 to satisfy a common V&V product of interest, and the computing system 108 may recommend a modified subset of the digital engineering tools 102 to the user 104 for final approval by the user 104, provided the user 104 is a qualified subject matter expert. After the subset of digital engineering tools 102 is identified, the computing system 108 may send instructions and / or input data to the identified subset of digital engineering tools 102 to perform one or more models, tests, and / or simulations. Results (or "engineering-related data output") of these models, tests, and / or simulations may be sent back and received at the computing system 108.

[0035] In still other implementations, user 104 may input a digital engineering tool (e.g., digital engineering tool 102F) needed to satisfy common V&V product 110I, and computing system 108 may determine that another digital engineering tool (e.g., digital engineering tool 102G) is also needed to satisfy common V&V product 110I. The computing system can then send instructions and / or input data to both digital engineering tools (e.g., digital engineering tools 102F and 102G), and the outputs of these digital engineering tools may be sent to and received at computing system 108. In some cases, the input data sent to one of the digital engineering tools (e.g., digital engineering tool 102G) may be derived (e.g., by computing system 108) from the output of another one of the digital engineering tools (e.g., digital engineering tool 102F).

[0036] After receiving the engineering-related data output from the digital engineering tool 102, the computing system 108 can process the received engineering-related data output to evaluate whether the requirements identified in the common V&V product of interest (e.g., Regulatory Standard 110E, Medical Standard 110G, Medical Certification Regulation 110H, Manufacturing Standard 110I, Manufacturing Certification Regulation 110J, etc.) are met. In some implementations, the computing system 108 can generate a report summarizing the results of the evaluation and transmit the report to the user device 106A or API 106B for review by the user 104. If all of the requirements are met, the prototype can be certified, resulting in a digitally certified product 112 (e.g., a digitally certified drug, chemical compound, or biological agent 112A, a digitally certified UAV 112B, a digitally certified manufacturing process 112C, etc.). However, if some of the regulatory requirements are not met, additional steps may need to be taken by the user 104 to certify the product prototype. In some implementations, the report sent to the user may include recommendations regarding these additional steps (e.g., suggesting one or more design changes, suggesting replacing one or more components with a previously designed solution, suggesting one or more adjustments to model, test, and / or simulation inputs, etc.). If the requirements of the common V&V product are partially met or exceed the collective capabilities of the distributed engineering tools 102, the computing system 108 may provide the user 104 with a report recommending partial certification, compliance, or satisfaction of a subset of the common V&V product (e.g., digital certification of a prototype subsystem or sub-process). The process of generating recommendations for the user 104 is described in further detail below.

[0037] In response to reviewing the report, the user 104 can make design changes to the digital prototype locally and / or send one or more instructions to the computing system 108 via the user device 106A or API 106B. These instructions may include, for example, instructions for the computing system 108 to reevaluate the updated prototype design, use one or more different digital engineering tools 102 for the evaluation process, and / or modify inputs to the digital engineering tools 102. The computing system 108 can then receive the user's instructions, perform one or more additional data manipulations in accordance with the instructions, and provide an updated report to the user 104. Through this iterative process, the user 104 can utilize the interconnected digital engineering and certification ecosystem 100 to design and ultimately certify prototypes (e.g., UAV prototypes, drug prototypes, manufacturing process prototypes, etc.) for common V&V products of interest (e.g., by providing certification compliance information). Importantly, because all of these steps are performed in the digital world (e.g., using digital prototypes, digital models / testing / simulations, and digital certification), significant amounts of time, cost, and materials may be saved compared to processes involving physical prototyping, evaluation, and / or certification of similar UAVs, drugs, manufacturing processes, etc. If the requirements associated with the common V&V product are partially met or exceed the collective capabilities of the digital engineering tools 102, the computing system 108 may provide a report to the user 104 recommending partial certification, compliance, or satisfaction of a subset of the common V&V product (e.g., digital certification of a prototype subsystem or sub-process).

[0038] While the above examples focus on the use of the interconnected digital engineering and authentication ecosystem 100 by a single user, further benefits of the ecosystem 100 may be realized through repeated use of the ecosystem 100 by multiple users. As described above, the centralized positioning of the computing system 108 within the architecture of the ecosystem 100 enables the computing system 108 to monitor and store various data flows through the ecosystem 100. Thus, as a growing number of users utilize the ecosystem 100 for digital product development, data associated with each use of the ecosystem 100 can be stored (e.g., in storage 118) and analyzed to yield various insights that can be used to further automate the digital product development process and make it easier to navigate for subject non-experts.

[0039] Indeed, in some implementations, the user's 104 user credentials may indicate the user's 104 skill level, allowing the user to control the amount of automated assistance provided. For example, a subject non-expert may only be allowed to utilize the ecosystem 100 to view pre-built designs and / or solutions, use the digital engineering tools 102 with certain default parameters, and / or follow a predetermined workflow with automated assistance guiding the user 104 through the product development process. On the other hand, a more skilled user may still be provided with automated assistance, but may be provided with more opportunities to override the default or suggested workflows and settings.

[0040] In some implementations, the computing system 108 may host applications and services 122 that automate or partially automate components of common V&V products, expected or common data transmissions, including data transmission components, from users 104, expected or common interfaces and / or data exchanges, including interface components, between various digital engineering tools 102, expected or common interfaces and / or data exchanges, including interface components, with machine learning models implemented on the computing system 108 (e.g., models trained and / or implemented by the machine learning engine 120), and expected or common interfaces and / or data exchanges between the applications and services themselves (e.g., in the applications and services layer 122).

[0041] In some implementations, data (or portions of the data) from multiple uses of ecosystem 100 may be aggregated to develop a training dataset, which may then be used to train a machine learning model (e.g., using machine learning engine 120) to perform various tasks, including identifying which of digital engineering tools 102 should be used to satisfy a particular common V&V product, identifying specific models, tests, and / or simulations (including inputs thereto) to be performed using digital engineering tools 102, identifying common V&V products that need to be considered for a particular type of product, identifying one or more recommended actions for a user 104 to take in response to an unsatisfied regulatory requirement, estimating the sensitivity of a model / test / simulation to particular inputs, etc. The output of the trained machine learning model may be used to implement various features of the interconnected digital engineering and certification ecosystem 100, including automatically suggesting inputs (e.g., inputs to the digital engineering tool 102) based on previously entered inputs, predicting time and cost requirements for developing a product, predictively estimating the results of sensitivity analyses, and even suggesting (e.g., by assistive or generative AI) design modifications, original designs, or design alternatives to a user's prototype to overcome one or more requirements (e.g., regulatory and / or certification requirements) associated with the common V&V product. In some implementations, with sufficient training data, the machine learning engine 120 may independently generate new designs, models, simulations, tests, and / or common V&V products based on data collected from multiple uses of the ecosystem 100.

[0042] In addition to storing usage data to enable the development of machine learning models, previous prototype designs and / or solutions (e.g., previously designed components, systems, models, simulations, and / or other engineering representations thereof) may be stored within ecosystem 100 (e.g., in storage 118) to enable users to search for and build on the work of others. For example, previously designed components, systems, models, simulations, and / or other engineering representations thereof can be searched for by user 104 and / or suggested to user 104 by computing system 108 to satisfy one or more requirements related to a common V&V product. The previously designed components, systems, models, simulations, and / or other engineering representations thereof can be used by user 104 as is or can be used as a starting point for additional modifications. This store (repository) of previously designed components, systems, models, simulations, and / or other engineering representations thereof (whether or not they are ultimately certified) can be monetized to create a marketplace of digital products that can be utilized to save time during the digital product development process, inspire users with alternative design ideas, avoid duplication of effort, and more. In some implementations, data corresponding to previous designs and / or solutions may be stored only if the users who developed the designs and / or solutions choose to share the data. In some implementations, a repository of previous designs and / or solutions may be containerized for private use (e.g., to avoid unwanted disclosure of proprietary information) within a single company, team, organization, or technology field.In some implementations, user credentials associated with the user 104 may be checked by the computing system 108 to determine which designs and / or solutions stored in the repository may be accessed by the user 104. In some implementations, use of previously designed components, systems, models, simulations, and / or other engineering representations thereof may be available only to other users who pay a usage fee.

[0043] 2A-2B, an example of an exemplary digital product development and certification workflow 200 is shown that may be implemented using the integrated digital engineering and certification ecosystem 100 (shown in FIG. 1). While not intended to be limiting, the workflow 200 is used to illustrate an illustrative and practical example showing the types of workflows enabled by the ecosystem 100 and its various features. In FIGS. 2A-2B, the individual steps of the workflow 200 are grouped by the elements of the ecosystem 100 that perform them (i.e., the user device 106A or API 106B operated by the user 104, the computing system 108 connected to (and / or including) the storage 118, the machine learning engine 120, and the application and services layer 122, the digital engineering tools 102, and the repository of the common V&V product 110).

[0044] In step 202, the user 104 can upload an MBSE file corresponding to a digital representation of a product (e.g., a UAV) from the user device 106A or API 106B to the computing system 108. In step 204, the user 104 can also upload a CAD file corresponding to the digital representation of the product from the user device 106 to the computing system 108.

[0045] The computing system 108 can receive the MBSE file (206), process the MBSE file to extract weight requirements (208), and send the data to the MBSE tool (210). For example, the data sent to the MBSE tool can include updated weight data, and the computing system 108 can request the MBSE tool to update the MBSE file with the updated weight data. The request can be made, for example, via the API 114 shown in FIG. 1.

[0046] Similarly, the computing system 108 can receive 212 a CAD file, process the CAD file to, for example, calculate mass properties of the digital prototype 214, and send 216 the data to the CAD tool. For example, the data sent to the CAD tool can include identified weight issues, and the computing system 108 can request the CAD tool to update the CAD file with the identified weight issues (e.g., by highlighting the identified issues in the CAD file). This request can also be made, for example, via the API / SDK 114 shown in FIG. 1.

[0047] In step 218, the MBSE tool (e.g., one of the digital engineering tools 102) can receive the data sent to it from the computing system 108. The MBSE tool can then update (220) the weight data in the MBSE file and export (222) the updated MBSE file to the computing system 108.

[0048] Similarly, in step 224, a CAD tool (e.g., another one of the digital engineering tools 102) can receive the data sent to the CAD tool from the computing system 108. The CAD tool can then update the CAD file to highlight any issues in the CAD file (226) and export the updated CAD file to the computing system 108 (228).

[0049] In step 230, the computing system 108 may receive the updated CAD and MBSE files exported by the CAD and MBSE tools, respectively.

[0050] In step 232, the computing system 108 may transmit (232) a request for data (e.g., regulatory and / or certification data) corresponding to one or more common V&V products. For example, the request may be transmitted via the API / SDK 116 illustrated in FIG. 1 to be processed in the repository of common V&V products 110, which may be off-the-shelf and / or hosted by a certification authority or another third party, as described above. Although step 232 is illustrated in workflow 200 after the computing system 108's communication with the digital engineering tool 102 (e.g., steps 210, 216, 222, 228, 230), in some implementations, the computing system 108 may retrieve data corresponding to one or more common V&V products from the repository of common V&V products 110 prior to the digital engineering tool 102's communication. 2A-2B, the data requested in step 232 may correspond to "MIL HDBK 516c 5.5.2 (JSSG-2006)"—a regulatory standard that defines the weight and center of gravity requirements for an aircraft to be certified as "airworthy." The repository of common V&V products 110 receives (236) a request for data corresponding to one or more common V&V products and transmits (238) the corresponding data to computing system 108, which in turn may receive (240) the data (e.g., regulatory and / or certification data) corresponding to the one or more common V&V products.

[0051] In step 242, the computing system 108 may process the updated CAD and MBSE files (received in step 230) and the data corresponding to one or more common V&V products (received in step 240) to identify and evaluate one or more requirements for certification. For example, the computing system may automatically process the CAD and MBSE files to calculate the weight and center of gravity of the physical manifestation of the digital prototype. The computing device may then compare these to the aircraft weight and center of gravity requirements identified in the data corresponding to MIL HDBK 516c 5.5.2 (JSSG-2006).

[0052] In step 244, based on the computing system's evaluation of the requirements identified in the data corresponding to "MIL HDBK 516c 5.5.2 (JSSG-2006)," the computing system 108 may generate and send a report to the user device 106A or the API 106B. The report may summarize the results of the evaluation, including an indication of whether the identified requirements were met. In some implementations, the report may also include one or more recommended actions for the user. The recommendations may be generated using, for example, the machine learning engine 120, as previously described above in connection with FIG. 1.

[0053] At step 246, the user device 106A or the API 106B can receive the report, and the user 104 can review the report. For example, the report can be presented on a display of the user device 106A for review by the human user 104A and / or received at the API 106B for processing by the artificial user 104B. In response to reviewing the report, the user 104 can operate the user device 106A or the API 106B to update the prototype design and / or send data representing one or more user instructions to the computing system 108 (248). As mentioned above, such user instructions may include instructions for the computing system 108 to reevaluate the updated prototype design, use one or more different digital engineering tools 102 for the evaluation process, and / or modify inputs to the digital engineering tools 102. At step 250, the computing system 108 can receive the one or more user instructions and perform one or more data operations in accordance with the one or more user instructions. In this way, the workflow 200 may enable the iterative design of digital prototypes using the interconnected digital engineering and certification ecosystem 100 to design and certify products such as UAVs, pharmaceuticals, manufacturing processes, etc., entirely within the digital world.

[0054] Referring now to FIG. 3 , a series of exemplary displays 300 are shown on a user device 106A. It is noted that in implementations including an artificial user 104B interfacing with a computing system via an API 106B, no display is necessary because the artificial user 104B can directly process digital computer files received at the API 106B without further visualization. The series of exemplary displays 300 may correspond to the exemplary workflow 200 described in connection with FIGS. 2A-2B. Again, these displays are not intended to be limiting but merely indicative of the kind of user experience a user 104 (and particularly a human user 104A) may encounter while using the interconnected digital engineering and certification ecosystem 100 for digital product development. The series of exemplary displays 300 described herein highlight the ease of use of the ecosystem 100 and the avoided complexity of requiring a user to separately interface with individual digital engineering tools and manually review complex common V&V products to evaluate whether a product prototype should be certified.

[0055] Display 302 illustrates a login screen that may be displayed on user device 106A. The login screen may prompt user 104 to enter user credentials (e.g., username and password) to access computing system 108 and the rest of the interconnected digital engineering and authentication ecosystem 100. The user credentials associated with user 104 may serve a variety of functions. For example, as described above, the user credentials may be associated with the user's skill level, which may control which features of ecosystem 100 the user 104 can access. In some implementations, the user credentials may additionally or alternatively be associated with the user's affiliation (e.g., to a particular company and / or organizational entity), which may determine the previously designed products and / or solutions the user may search for and / or be suggested by computing system 108. Generally, the user credentials may help ensure that the user 104 can access only information within ecosystem 100 that the user 104 is entitled to and / or authorized to access.

[0056] Once user 104 logs in from user device 106A, user device 106A may be used to develop a digital prototype of a product. For example, display 304 shows a modeling screen that user 104 might see while developing a digital model of a UAV (e.g., using a CAD tool). Once the prototype is developed, the user can upload prototype data, such as CAD files and / or MBSE files, to computing system 108 (e.g., as in steps 202 and 204 of workflow 200). Thus, display 306 shows a screen that may prompt user 104 to upload the MBSE files and CAD files to computing system 108.

[0057] Once the user uploads the MBSE file and the CAD file to the computing system 108, the computing system 108 may perform several steps (e.g., steps 206, 207, 210, 212, 214, 216, 230, 232, 240, 242, 244 of workflow 200) to evaluate the prototype with respect to one or more requirements identified in the common V&V product and generate a report summarizing the evaluation. In doing so, the computing system 108 may communicate with the digital engineering tools 102 and the repository of the common V&V product 110, which may themselves perform operations (e.g., steps 218, 220, 222, 224, 226, 228, 236, 238 of workflow 200) to facilitate evaluation of the prototype. These steps take time to complete (e.g., ranging from a few seconds to a few hours), during which time a display 308 may be shown on the screen of the user device 106A, providing information about the current status of the prototype's evaluation.

[0058] Once the evaluation of the prototype is complete (e.g., in step 244 of workflow 200), a generated report may be sent from computing system 108 to user device 106A. Display 310 shows a screen of user device 106A presenting the report to user 104. The report may present information indicating whether one or more requirements identified within the common V&V product of interest were met, and may also present information about one or more issues (e.g., problematic components of the device) that resulted in the unsatisfied requirement. In some implementations, the presented information may also include more detailed data from the evaluation and / or proposed solutions for resolving one or more issues to meet the requirements. The easy-to-understand format of the report presented in display 310 can assist user 104 in understanding why the prototype may not meet one or more requirements and can provide user 104 with actionable suggestions for improving the digital prototype. Even in implementations that include an artificial user 104B (where a screen display is not required), a concise or standardized report in the form of a digital computer file sent to the API 106B can similarly assist the artificial user 104B in understanding why the prototype may not satisfy one or more requirements and can provide the artificial user 104B with actionable suggestions for improving the digital prototype.

[0059] Referring now to FIG. 4, a flow diagram 400 is depicted illustrating an exemplary product design process using the interconnected digital engineering and certification ecosystem 100. By translating elements and characteristics, such as physical laws, from the physical world 402 to the digital world 406 via transfer function models and tools 404, product development and certification may be enabled entirely (or nearly entirely or partially) within the ecosystem 100. It is noted that the entire physical world may not be perfectly replicated at once; in some cases, a very specific subset of the physical world may be described digitally by any number of models or simulations (e.g., a model describing air turbulence between 200 and 600 knots complemented by another model describing air turbulence between 500 and 800 knots), which together amount to a sufficient representation of the physical world to enable certification of a particular system or product. An interconnected digital engineering and certification ecosystem (e.g., ecosystem 100) can allow digital representations of the physical world to be constructed in a modular and complementary manner, allowing each of these pieces to be added individually by many different people and entities, without the need for coordination and with aligned incentives (e.g., enabled by intellectual property protection, monetization of models or valuable digital representations of the physical world, etc.), allowing each of the models or simulations to connect to and build upon each other. For example, digital product development 408, digital product testing 410, and digital product certification 412 can all occur within ecosystem 100, occurring entirely within the digital realm or “metaverse” to produce a finalized digital product design 414. Following this product design process, the finalized digital product design need only be transformed into the physical world (e.g., by manufacturing 416) at the very end of the product design process to produce a final physical-world product 418.This is in stark contrast to current product development and certification workflows that utilize digital engineering tools but may still often require the physical fabrication, testing, and certification of prototypes throughout the iterative product development, product testing, and product certification process. Compared to such workflows, a product design process enabled by an interconnected digital engineering and certification ecosystem 100 can therefore result in significant savings in time, cost, materials, and environmental impact.

[0060] 5, the interconnected digital engineering and authentication ecosystem 100 provides a variety of different opportunities for monetization (shown as blocks 500A-500D). In some implementations, interactions between a user 104 and a computing system 108 may include a monetization opportunity 500A. For example, a user 104 may be charged for sending instructions to the computing system 108 and / or a user 104 may be charged for downloading data (e.g., authentication reports) from the computing system 108. Fees may be subscription-based (e.g., charging a monthly or annual fee to use the computing system 108), usage-based (e.g., charging the user 104 based on the number of interactions with the computing system 108, the amount of time spent interacting with the computing system 108, etc.), or hybrid (e.g., using a freemium model).

[0061] In some implementations, the interaction between the computing system 108 and the digital engineering tool 102 may include monetization opportunities 500B. For example, the user 104 may be charged for transmitting data between the computing system 108 and / or the digital engineering tool 102. In some implementations, the fee paid by the user 104 may be split between the third-party provider of the digital engineering tool 102 and the operator of the computing system 108. In some implementations, the third-party provider of the digital engineering tool 102 may itself pay a fee to the operator of the computing system 108 to have their digital engineering tool included in the ecosystem 100. Fees for users 104 can be subscription-based (e.g., charging a monthly or annual fee to access a particular digital engineering tool 102), usage-based (e.g., charging users 104 based on the amount of data transferred between the digital engineering tool 102 and the computing system 108, the amount of processing time required by the digital engineering tool 102, etc.), or mixed (e.g., using a freemium model).

[0062] In some implementations, the interaction between the computing system 108 and the repository of common V&V products 110 may include monetization opportunities 500C. For example, the user 104 may be charged to transmit data between the computing system 108 and / or the repository of common V&V products 110. In some implementations, the fee paid by the user 104 may be split between the authority operating the repository of common V&V products 110 and the person operating the computing system 108. The fee for the user 104 may be subscription-based (e.g., charging a monthly or annual fee to access the repository of common V&V products 110), usage-based (e.g., charging the user 104 based on the amount of data transferred between the repository of common V&V products 110 and the computing system 108, the number of common V&V products requested, etc.), or hybrid (e.g., using a freemium model).

[0063] In some implementations, the final authentication of the digitally authenticated product 112 by the computing system 108 may also include a monetization opportunity 500D. For example, the user 104 may be charged a fee to perform a formal authentication of the user's product. Additionally or alternatively, the user 104 may be charged a fee to download proof of authentication.

[0064] 8, an example architecture of a digital engineering (DE) platform 800 (such as interconnected digital engineering and authentication ecosystem 100) is shown. The example architecture of digital engineering platform 800 is designed according to zero trust security principles and is further designed to support scalability, as well as robust and resilient operation.

[0065] In one embodiment, the architecture of digital engineering platform 800 includes multiple components: digital engineering (DE) platform enclave 802, cloud services 804, and customer environment 810. Customer environment 810 optionally includes DE platform exclave 816.

[0066] The DE platform enclave 802 can serve as the launch pad for services provided by the platform 800. The enclave 802 can be viewed as a central command hub responsible for managing operations and functions. For example, the enclave 802 can be implemented using the computer systems 108 of the interconnected digital engineering and authentication ecosystem 100 described above. The DE platform enclave 802 serves as a centralized command and control hub responsible for orchestrating and managing all platform operations. The DE platform enclave 802 is designed to integrate both a zero-trust security model and hyperscale capabilities, resulting in a secure and scalable processing environment tailored to individual customer needs. Zero-trust security features include, but are not limited to, strict access control, algorithmic fairness, and data isolation. The enclave 802 also supports a machine learning engine (e.g., the machine learning engine 120) for real-time analytics, auto-scaling features for workload adaptability, and API-based interoperability with third-party services. Security and resource optimization are enhanced through support for multi-tenancy, role-based access control, and data encryption both at rest and in transit. Digital engineering platform enclave 802 may also include one or more of the features described below.

[0067] First, the digital engineering platform enclave 802 may be designed according to zero trust security principles. In particular, the DE platform enclave 802 employs zero trust principles to ensure that no implicit trust is assumed between any elements within the system, such as digital models, platform agents, or individual users (e.g., users 104A, 104B), or their actions. The model is further reinforced by strict access control mechanisms that restrict even administrative teams (e.g., teams of individuals associated with the platform provider) to predetermined, limited access to the enclave's resources. To reinforce this strong security stance, data encryption is applied both at rest and in transit, effectively mitigating the risk of unauthorized access and data breaches.

[0068] The DE platform enclave 802 can also be designed to maintain isolation and independence. A key aspect of the enclave's architecture is its emphasis on fairness and isolation. The enclave 802 does not allow cryptographic dependencies from external enclaves and enforces strong isolation policies. The enclave design also enables both single-tenant and multi-tenant configurations, further strengthening data and process isolation between customers 806 (e.g., users 104A, 104B). Furthermore, the enclave 802 is designed with decoupled resource sets to minimize interdependencies, thereby increasing system efficiency and autonomy.

[0069] DE platform enclave 802 may be designed for scalability and adaptability. Enclave 802 is engineered to be scalable and adaptable to better accommodate a variety of operational requirements. For example, enclave 802 may incorporate hyperscale-like characteristics in conjunction with zero-trust principles to enable scalable growth and effectively handle high-performance workloads.

[0070] DE platform enclave 802 may further be designed for workflow adaptability accommodated by strict access control mechanisms. DE platform enclave 802 is designed to accommodate various customer workflows and DE models through its strict access control mechanisms. This configurability enables a modular approach to integrating different functions, from data ingestion to algorithm execution, without compromising zero-trust security posture. Platform 800's adaptability gives it high versatility for many use cases while ensuring stable performance and robust security.

[0071] DE platform enclave 802 may be further designed to enable analytics for robust platform operation. At the core of the enclave's operational efficiency is a machine learning engine (e.g., machine learning engine 120) capable of performing real-time analytics. This improves decision-making and operational efficiency throughout platform 800. Auto-scaling mechanisms may also be included to enable dynamic resource allocation based on workload demands, further increasing the responsiveness and efficiency of the platform.

[0072] In an example implementation, the DE platform enclave 802 may include several components, as shown in FIG. 8 and described in further detail herein.

[0073] In the embodiment of DE platform enclave 802 shown in Figure 8, DE platform enclave 802 includes a "Monitoring Service" and a "Telemetry Service" as part of a "Monitoring Service Cell." These components focus on maintaining, tracking, and analyzing the performance of platform 800 to ensure optimal service delivery, including advanced machine learning capabilities for real-time analysis.

[0074] In the embodiment of DE platform enclave 802 shown in FIG. 8, DE platform enclave 802 also includes a “Static Assets Service Cell” that houses the platform's 800 user interface, SDK, command line interface (CLI), and documentation.

[0075] In the embodiment of the DE platform enclave 802 shown in FIG. 8 , the DE platform enclave 802 includes the DE platform APIs (e.g., APIs 114, 116) and further includes an “API gateway service cell” that acts as an intermediary for requests between client applications (e.g., digital engineering tools 102, repository of common V&V products 110, etc.) and platform services.

[0076] 8, DE platform enclave 802 further includes a "search service cell." This component facilitates efficient retrieval of information from DE platform 800 and enhances the overall functionality of DE platform 800.

[0077] 8, DE platform enclave 802 further includes a “logging service cell” and a “control plane service cell.” These components help record and manage operational events and the flow of information within platform 800.

[0078] As shown in FIG. 8 , the digital engineering platform 800 architecture also includes cloud services 804, which may not interact with customer data but may include services that can modify software for orchestrating the operation of the digital engineering platform. In an exemplary implementation, several cloud resources provide support and infrastructure services to the platform. For example, in the embodiment of DE platform 800 shown in FIG. 8 , cloud services 804 include “Customer IAM Services,” where “IAM” stands for “Identity and Access Management.” The identity and access management services ensure secure and controlled access to platform 800.

[0079] In the embodiment of the DE platform 800 shown in FIG. 8, the cloud services 804 also include a "test service" that includes testing tools for validating the operation of the platform.

[0080] In the embodiment of DE platform 800 shown in FIG. 8, cloud services 804 also include an “orchestration service” for controlling and managing the lifecycle of containers on platform 800.

[0081] 8, cloud services 804 also include "artifact services" and "version control and build services." These cloud services are critical to maintaining the progress of projects, code, and instances within the system while also managing the artifacts generated during the product development process.

[0082] As shown in FIG. 8 , the digital engineering platform 800 architecture also includes a customer environment 810 with an “Authoritative Source of Truth” 812, customer tools 814, and an optional DE platform enclave 816. The customer environment 810 is where customer data resides and is processed in a zero-trust manner by the digital engineering platform 800. As previously mentioned, the DE platform enclave 802 provides a robust and scalable environment for secure processing of critical workloads according to the customer's unique needs, focusing on both zero-trust principles and hyperscale-like characteristics. In some examples, the DE platform exclave 816 resides within the customer environment 810 to support the customer's 806 digital engineering tasks and operations.

[0083] When a customer 806 (e.g., user 104A, 104B) intends to perform a digital engineering task using the digital engineering platform 800 (e.g., the interconnected digital engineering and authentication ecosystem 100), typical operations include secure data ingestion and controlled data retrieval. Derived data generated through digital engineering operations, such as updated digital model files or revisions of digital model parameters, is stored solely within the customer environment 810, and the digital engineering platform 800 may provide tools for accessing the derived data's metadata. An exemplary implementation may include secure data ingestion utilizing zero trust principles to ensure customer data is securely uploaded to the customer environment 810 through a pre-validated secure tunnel, such as a Secure Sockets Layer (SSL) tunnel. This may enable secure file transfer directly to designated cloud storage, such as an S3 bucket, within the customer environment 810. An exemplary implementation may also include controlled data retrieval, in which temporary, pre-authenticated URLs generated via a secure token-based mechanism are used for controlled data access, thereby minimizing the risk of unauthorized interactions. Exemplary embodiments may also include immutable derived data, such that transformed data generated through operations such as data extraction is securely stored within customer environment 810 while adhering to zero trust security protocols. Exemplary embodiments may also include a tokenization utility, where a specialized digital engineering (DE) platform tool called a "tokenizer" is deployed within customer environment 810 for secure management of derived metadata that conforms to zero trust guidelines.

[0084] The customer environment 810 interacts with other elements of the secure digital engineering (DE) platform 800 and includes several features that handle data storage and secure interaction with the platform 800. For example, one element of the customer environment 810 is the "trusted authoritative source" 812, which is the main repository of customer data, ensuring data integrity and accuracy. Nested within this are "customer buckets," where data is securely stored with strict access controls that restrict access to the data to authorized users or processes through pre-authenticated URL links. This setup ensures uncompromising data security within the customer environment 810 while providing smooth interaction with other elements of the DE platform 800.

[0085] The customer environment 810 also includes additional software tools (e.g., customer tools 814) that may be utilized based on specific customer requirements. For example, a "DE tool host" is a component that handles the necessary data engineering applications for working with customer data. The "DE tool host" includes a DET CLI (Data Engineering Tool Command Line Interface) that enables user-friendly command-line operation of the DE tools (e.g., digital engineering tools 102). A "DE platform agent" ensures smooth communication and management between the customer environment 810 and elements of the DE platform 800. Additionally, there may be another set of optional DE tools designed to support customer-specific data engineering workflows.

[0086] In some cases, an optional feature known as the "DE Platform Exclave" 816 may be used within the customer environment 810 for enhanced security. The Exclave 816 operates within the customer's network and oversees data processing while providing hyperscale-like platform performance and strict adherence to zero trust principles. The Exclave 816 includes a "DE Tool Host" that runs the DE tools and agents required for its operation.

[0087] 6, an example process 600 for product development is shown. In some implementations, process 600 may be performed by a computing system (e.g., computing system 108) of an interconnected digital engineering and certification ecosystem (e.g., ecosystem 100).

[0088] Operations of process 600 include receiving 602 design and / or engineering data (D / E data) corresponding to a prototype representation of a product from a user device. For example, the user device can correspond to user device 106A or API 106B, and the D / E data can correspond to MBSE files, CAD files, and / or other digital files or information related to the digital prototype, as described above. In some implementations, the product can be a UAV or other type of aircraft, an automobile, a boat, a submersible vehicle, an industrial robot, a spacecraft, a satellite, a structure, a tool, a physical device, a mobile device, a drug, a chemical product, or a biological agent, a manufacturing process, or any other complex system (either physical or non-physical) that may be evaluated for a common V&V product.

[0089] Operations of process 600 also include sending one or more inputs derived from the D / E data to one or more digital engineering tools for processing (604). For example, the one or more digital engineering tools may correspond to digital engineering tool 102 described above. In some implementations, at least a subset of the one or more digital engineering tools may include model-based systems engineering (MBSE) tools, augmented reality (AR) tools, computer-aided design (CAD) tools, data analytics tools, modeling and simulation (M&S) tools, product lifecycle management (PLM) tools, simulation engines, requirements models, electronics models, test planning models, cost models, schedule models, software modeling, supply chain models, manufacturing models, cybersecurity models, multi-attribute tradespace tools, or mission effects models, or other similar digital engineering tools that may be recognized by those skilled in the art as engineering design tools.

[0090] Operations of process 600 also include receiving 606 engineering-related data output from one or more digital engineering tools. For example, the engineering-related data output may correspond to results of models, tests, and / or simulations performed by data engineering tool 102, as described above.

[0091] Operations of process 600 also include receiving (608) data corresponding to one or more common V&V products associated with the product. For example, the one or more common V&V products can be digitized regulatory and / or certification standards and may correspond to common V&V products 110A-110J stored in the repository of common V&V products 110 described above. In some implementations, the data corresponding to the one or more common V&V products may be received from a user device (e.g., by user upload). In some implementations, the data corresponding to the one or more common V&V products may be received from a regulatory and / or certification authority (e.g., via a repository of common V&V products hosted or maintained by the regulatory and / or certification authority).

[0092] Operations of process 600 also include identifying 610 one or more requirements for the product based on data corresponding to one or more common V&V products. For example, the one or more requirements may correspond to requirements that must be met to certify the product according to a particular common V&V product.

[0093] The operations of process 600 also include determining (612) whether one or more requirements are satisfied based on the engineering-related data output and data corresponding to the one or more common V&V products. In some implementations, rather than making a binary determination, the operations of process 600 may include determining whether one or more requirements are likely to be satisfied by the prototype representation of the product (e.g., based on an estimated probability). In some implementations, determining whether one or more requirements are satisfied (or whether they are likely to be satisfied) based on the engineering-related data output may include determining whether one or more requirements are satisfied with or without human input.

[0094] Operations of process 600 also include presenting (614) at the user device information corresponding to the engineering-related data output and / or data corresponding to the one or more common V&V products, where the presented information includes an indication of whether one or more requirements are satisfied. In some implementations, the presented information may include an indication of the probability of whether the one or more requirements are satisfied by the prototype representation of the product. For example, the information may be presented at the user device in the form of a report, as shown in display 310 of FIG. 3 and as described above. In some implementations, the presented information may further include recommended actions that a user of the user device can take to satisfy the one or more requirements. In such implementations, the recommended actions may include a suggestion to use a particular one of the one or more digital engineering tools, a suggestion to modify one or more inputs submitted to the one or more digital engineering tools, a suggestion to modify one or more components of the prototype representation of the product, a suggestion to replace one or more components of the prototype representation of the product with a previously designed solution, and / or a suggestion of a completely or partially new design generated by the system (e.g., using machine learning engine 120).

[0095] The process operations also include receiving (616) instructions from the user device after presenting information corresponding to the engineering-related data output and / or data corresponding to the one or more common V&V products at the user device, the instructions corresponding to one or more user interactions with the user device.

[0096] The operation of the process also includes performing one or more operations on the D / E data in response to receiving instructions from the user device (618). In some implementations, performing one or more operations on the D / E data may include modifying the D / E data and / or deriving modified inputs from the D / E data for transmission to one or more digital engineering tools.

[0097] Additional operations of process 600 may include the following. In some implementations, process 600 may include storing, in a storage device, usage data representing received data corresponding to one or more common V&V products, received D / E data, engineering-related data output from one or more digital engineering tools, an indication of whether one or more requirements have been met (or are likely to be met), one or more user interactions with a user device, and / or one or more manipulations of the D / E data. Process 600 may also include incorporating applications and services (e.g., applications and services 122) that automate or partially automate the determination of whether one or more requirements have been met or partially met. Process 600 may further include incorporating at least a portion of the usage data into a training dataset and training a machine learning model based on the training dataset. In some implementations, the machine learning model may be configured to receive as input information about another product being designed by another user and output a suggestion for the other user to use a particular one of the one or more digital engineering tools, a suggestion to modify one or more inputs submitted by the other user to the one or more digital engineering tools, a suggestion to modify one or more components of another prototype representation associated with the other user, and / or a suggestion to replace one or more components of the other prototype representation with a previously designed solution. In some implementations, the process 600 may also include using the stored usage data for one or more sensitivity analyses. In some implementations, the process 600 may also include using the stored usage data to improve the performance of applications and services (e.g., applications and services 122).

[0098] In some implementations, additional operations of process 600 may include checking one or more credentials of the user before performing one or more operations on the D / E data, and determining that the user is entitled to or authorized to perform one or more operations on the D / E data based on the one or more credentials.

[0099] FIG. 7 illustrates examples of a computing device 700 and a mobile computing device 750 used to implement the present disclosure. Computing device 700 is intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other suitable computers. Mobile computing device 750 is intended to represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, VR devices, and other similar computing devices. The components, their connections and relationships, and their functions illustrated herein are intended to be exemplary only and not limiting. Computing device 700 and / or mobile computing device 750 may form at least a portion of user device 106A or API 106B and computing system 108 described above. As previously mentioned, in some implementations, computing system 108 can be a distributed computing system including multiple computing devices, such as computing device 700 and / or mobile computing device 750. In other implementations, computing system 108 may include a single computing device. In some implementations, API 106B may be implemented on computing device 700 and / or mobile computing device 750 to relay digital computer files to a non-human artificial user 104B (e.g., an artificial intelligence and / or algorithmic user), which may itself be implemented on computing device 700 and / or mobile computing device 750 (or in a separate instance of computing device 700 and / or mobile computing device 750).

[0100] Computing device 700 includes processor 702, memory 704, storage device 706, high-speed interface 708, and low-speed interface 712. In some implementations, high-speed interface 708 connects to memory 704 and multiple high-speed expansion ports 710. In some implementations, low-speed interface 712 connects to low-speed expansion port 714 and storage device 706. Each of processor 702, memory 704, storage device 706, high-speed interface 708, high-speed expansion port 710, and low-speed interface 712 are connected to each other using various buses and may be mounted on a common motherboard or otherwise as needed. Processor 702 can process instructions for execution within computing device 700, including instructions stored in memory 704 and / or on storage device 706, to display graphical information for a graphical user interface (GUI) on an external input / output device, such as a display 716 coupled to high-speed interface 708. In other implementations, multiple processors and / or multiple buses may be used, along with multiple memories and multiple types of memory, as appropriate. Additionally, multiple computing devices may be connected (eg, as a server bank, a group of blade servers, or a multi-processor system) with each device performing a portion of the required operations.

[0101] The memory 704 stores information within the computing device 700. In some implementations, the memory 704 is a volatile memory unit or multiple volatile memory units. In some implementations, the memory 704 is a non-volatile memory unit or multiple non-volatile memory units. The memory 704 may also be another form of computer-readable medium, such as a magnetic or optical disk.

[0102] Storage device 706 can provide mass storage for computing device 700. In some implementations, storage device 706 can be or include a computer-readable medium such as a floppy disk device, a hard disk device, an optical disk device, a tape device, a flash memory, or other similar solid-state memory device, or an array of devices, including devices in a storage area network or other configuration. Instructions can be stored on an information carrier. When executed by one or more processing devices, such as processor 702, the instructions perform one or more methods, such as those described above. The instructions can also be stored by one or more storage devices, such as a computer-readable or machine-readable medium, such as memory 704, storage device 706, or memory on processor 702.

[0103] The high-speed interface 708 manages bandwidth-intensive operations for the computing device 700, while the low-speed interface 712 manages less bandwidth-intensive operations. Such an allocation of functionality is merely exemplary. In some implementations, the high-speed interface 708 is coupled to memory 704, to a display 716 (e.g., through a graphics processor or accelerator), and to a high-speed expansion port 710, which may accept various expansion cards. In implementations, the low-speed interface 712 is coupled to a storage device 706 and a low-speed expansion port 714. The low-speed expansion port 714, which may include various communication ports (e.g., Universal Serial Bus (USB), Bluetooth, Ethernet, Wireless Ethernet), may be coupled to one or more input / output devices. Such input / output devices may include a scanner 730, a printing device 734, or a keyboard or mouse 736. The input / output devices may also be coupled to the low-speed expansion port 714 through a network adapter 732. Such network input / output devices may include, for example, a switch or a router.

[0104] Computing device 700 may be implemented in many different forms, as shown in FIG. 7 . For example, computing device 700 may be implemented as a single standard server 720 or multiple times within a cluster of such servers. Furthermore, computing device 700 may be implemented in a personal computer, such as a laptop computer 722. Computing device 700 may also be implemented as part of a rack server system 724, a high-performance computing enclave, a quantum and / or non-silicon-based computing system. Alternatively, components of computing device 700 may be combined with other components of a mobile device, such as mobile computing device 750. Each such device may include one or more of computing device 700 and mobile computing device 750, and the entire system may be comprised of multiple computing devices communicating with each other.

[0105] The mobile computing device 750 includes, among other components, a processor 752, memory 764, an input / output device such as a display 754, a communication interface 766, and a transceiver 768. The mobile computing device 750 may also include a storage device, such as a microdrive or other device, to provide additional storage. Each of the processor 752, memory 764, display 754, communication interface 766, and transceiver 768 are interconnected using various buses, and some of the components may be mounted on a common motherboard or otherwise as needed. In some implementations, the mobile computing device 750 may include a camera device.

[0106] Processor 752 can execute instructions within mobile computing device 750, including instructions stored in memory 764. Processor 752 may be implemented as a chipset of chips including separate analog and digital processors. For example, processor 752 may be a complex instruction set computer (CISC) processor, a reduced instruction set computer (RISC) processor, or a minimal instruction set computer (MISC) processor. Processor 752 may coordinate other components of mobile computing device 750, such as, for example, controlling a user interface (UI), applications run by mobile computing device 750, and / or wireless communications by mobile computing device 750.

[0107] The processor 752 may communicate with a user through a control interface 758 and a display interface 756 coupled to a display 754. The display 754 may be, for example, a thin film transistor liquid crystal display (TFT) display, an organic light emitting diode (OLED) display, or other suitable display technology. The display interface 756 may include appropriate circuitry for driving the display 754 to present graphical and other information to the user. The control interface 758 may receive commands from the user and convert those commands for transmission to the processor 752. Additionally, an external interface 762 may provide communication with the processor 752 to enable near-field communication of the mobile computing device 750 with other devices. The external interface 762 may provide, for example, wired communication in some implementations or wireless communication in other implementations, and multiple interfaces may be used.

[0108] Memory 764 stores information within mobile computing device 750. Memory 764 may be implemented as one or more of a computer-readable medium or media, a volatile memory unit or units, or a non-volatile memory unit or units. Expansion memory 774 may also be provided and connected to mobile computing device 750 through expansion interface 772, which may include, for example, a single in-line memory module (SIMM) card interface. Expansion memory 774 may provide additional storage space for mobile computing device 750 or may store applications or other information for mobile computing device 750. In particular, expansion memory 774 may include instructions that perform or supplement the processes described above and may include secure information. Thus, for example, expansion memory 774 may be provided as a security module for mobile computing device 750 and may be programmed with instructions that enable secure use of mobile computing device 750. Additionally, secure applications may be provided by SIMM cards along with additional information, such as placing identifying information on the SIMM card in a way that cannot be hacked.

[0109] The memory may include, for example, flash memory and / or non-volatile random access memory (NVRAM), as discussed below. In some implementations, the instructions are stored on an information carrier. When executed by one or more processing devices, such as processor 752, the instructions perform one or more methods, such as those described above. The instructions may be stored by one or more storage devices, such as one or more computer-readable or machine-readable media, such as memory 764, expansion memory 774, or memory on processor 752. In some implementations, the instructions may be received in a propagated signal, such as via transceiver 768 or external interface 762.

[0110] The mobile computing device 750 may communicate wirelessly through a communication interface 766, which may include digital signal processing circuitry as needed. The communication interface 766 may provide communication under various modes or protocols, such as Global System for Mobile communications (GSM) voice calls, Short Message Service (SMS), Enhanced Messaging Service (EMS), Multimedia Messaging Service (MMS) messaging, Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), Personal Digital Cellular (PDC), Wideband Code Division Multiple Access (WCDMA), CDMA2000, and General Packet Radio Service (GPRS). Such communication may occur through a transceiver 768 using radio frequencies, for example. Additionally, short-range communication may occur, such as using Bluetooth or Wi-Fi. Additionally, a global positioning system (GPS) receiver module 770 may provide further navigation- and location-related wireless data to the mobile computing device 750 that may be used as appropriate by applications running on the mobile computing device 750.

[0111] The mobile computing device 750 may also communicate voice using an audio codec 760, which may receive spoken information from a user and convert that information into usable digital information. Similarly, the audio codec 760 may generate audible audio for the user, such as through a speaker on a handset of the mobile computing device 750. Such audio may include audio from a voice call, may include recorded audio (e.g., voice messages, music files, etc.), and may include audio generated by applications running on the mobile computing device 750.

[0112] The mobile computing device 750 may be implemented in many different forms, as shown in Figure 7. For example, the mobile computing device 750 may be implemented as a telephone device 780, a personal digital assistant 782, and a tablet device (not shown). The mobile computing device 750 may also be implemented as a component of a smartphone, an AR device, or other similar mobile device.

[0113] Computing device 700 and / or 750 may also include a USB flash drive, which may store an operating system and other applications. The USB flash drive may include input / output components such as a wireless transmitter or a USB connector that may be inserted into a USB port of another computing device.

[0114] Other embodiments and applications not specifically described herein are within the scope of the appended claims. Elements of different implementations described herein may be combined to form other embodiments. [Explanation of symbols]

[0115] 100 Interconnected Digital Engineering and Certification Ecosystem 102 Digital Engineering Tools 102A Data Analysis Tools 102B CAD and Finite Element Analysis Tools 102C Simulation Tool 102D~102E Pharmaceutical M&S Tools 102F~102G Manufacturing M&S Tools 104 users 104A Human Users 104B Artificial Users 106A User Device 106B API (or other similar machine-to-machine communication interface) 108 Computing Systems 110 Common V&V Products 110A-110F Regulatory standards related to the development and certification of UAVs 110G medical standard 110H Medical Certification Regulations 110I Manufacturing Standard 110J Manufacturing Certification Regulations 112, 112A~112C Digitally authenticated products 114 API / SDK 116 API / SDK 118 Data Storage Unit 120 Machine Learning Engine 122 Application and Service Layer 200 Digital Product Development and Certification Workflows 300 A series of illustrative displays 302 display 304 display 306 display 308 displays 310 displays 400 Flowchart 402 Physical World 404 Transfer Function Models and Tools 406 Digital World 408 Digital Product Development 410 Digital Product Testing 412 Digital Product Authentication 414 Finalized Digital Product Design 416 Manufacturing 418 End Products of the Physical World 500A Monetization Opportunities 500B Monetization Opportunities 500C Monetization Opportunity 500D Monetization Opportunities 600 processes 700 computing devices 702 processor 704 memory 706 Storage Devices 707 Storage Devices 708 high-speed interface 710 High-Speed ​​Expansion Port 712 Low-Speed ​​Interface 714 Low-Speed ​​Expansion Port 716 Display 720 Server 722 laptop computers 724 Rack Server System 730 Scanner 732 Network Adapter 734 Printing Device 736 keyboard or mouse 750 Mobile Computing Devices 752 processor 754 Display 756 Display Interface 758 Control Interface 760 audio codec 762 External Interface 764 memory 766 Communication Interface 768 Transceiver 770 GPS Receiver Module 772 Expansion Interface 774 Expanded Memory 780 phone device 782 Mobile Information Terminals 800 Digital Engineering Platform 802 DE Platform Enclave 804 Cloud Services 806 Customer 810 Customer environment 812 Trusted and Authoritative Sources 814 Customer Tools 816 DE Platform Exclave

Claims

1. receiving design and / or engineering data (D / E data) corresponding to a prototype representation of the product from a user device; transmitting one or more inputs derived from the D / E data to one or more digital engineering tools for processing; receiving engineering-related data output from the one or more digital engineering tools; receiving data corresponding to one or more common validation and verification (V&V) products associated with the product; identifying one or more requirements for the product based on the data corresponding to the one or more common V&V products; determining whether the one or more requirements are met based on the engineering-related data output and the data corresponding to the one or more common V&V products; presenting information at the user device corresponding to the engineering-related data output and / or the data corresponding to the one or more common V&V products, the presented information including an indication of whether the one or more requirements have been met or an indication of a probability of whether the one or more requirements are met by the prototype representation of the product; receiving instructions from the user device after presenting the information corresponding to the engineering-related data output and / or the data corresponding to the one or more common V&V products at the user device, the instructions corresponding to one or more user interactions with the user device; performing one or more operations on the D / E data in response to receiving the instruction from the user device; A computer-implemented method comprising:

2. The computer-implemented method of claim 1 , wherein the data corresponding to the one or more common V&V products is received from the user device.

3. 10. The computer-implemented method of claim 1, wherein the data corresponding to the one or more common V&V products is received from a regulatory and / or certification authority.

4. The computer-implemented method of claim 1 , wherein the product is an aircraft.

5. 10. The computer-implemented method of claim 1, wherein at least a subset of the one or more digital engineering tools comprises a model-based systems engineering (MBSE) tool, an augmented reality (AR) tool, a computer-aided design (CAD) tool, a data analytics tool, a modeling and simulation (M&S) tool, a product lifecycle management (PLM) tool, a simulation engine, a requirements model, an electronics model, a test planning model, a cost model, a schedule model, a software modeling, a supply chain model, a manufacturing model, a cybersecurity model, a multi-attribute tradespace tool, or a mission effects model.

6. 2. The computer-implemented method of claim 1, wherein determining whether the one or more requirements have been met based on the engineering-related data output comprises determining whether the one or more requirements have been met without any human input.

7. The computer-implemented method of claim 1 , wherein the presented information further includes recommended actions that the user of the user device can take to satisfy the one or more requirements.

8. 8. The computer-implemented method of claim 7, wherein the recommended actions include a suggestion to use a particular one of the one or more digital engineering tools, a suggestion to modify the one or more inputs sent to the one or more digital engineering tools, a suggestion to modify one or more components of the prototype representation of the product, and / or a suggestion to replace one or more components of the prototype representation of the product with a previously designed solution.

9. 2. The computer-implemented method of claim 1, wherein performing the one or more operations on the D / E data includes modifying the D / E data and / or deriving modified inputs from the D / E data for transmission to the one or more digital engineering tools.

10. 10. The computer-implemented method of claim 1, further comprising storing in a storage device usage data representing the received data corresponding to the one or more common V&V products, the received D / E data, the engineering-related data output from the one or more digital engineering tools, the indication of whether the one or more requirements have been met, the indication of the probability of whether the one or more requirements are met by the prototype representation of the product, the one or more interactions of the user with the user device, and / or the one or more manipulations of the D / E data.

11. incorporating at least a portion of the usage data into a training data set; training a machine learning model based on the training dataset; 11. The computer-implemented method of claim 10, further comprising:

12. The machine learning model: receiving as input information about another product being designed by another user; outputting a proposal for the other user to use a particular one of the one or more digital engineering tools, a proposal to modify one or more inputs submitted by the other user to the one or more digital engineering tools, a proposal to modify one or more components of another prototype representation associated with the other user, a proposal to replace one or more components of the other prototype representation with a previously designed solution, and / or a proposal for a completely or partially new design generated using a machine learning engine; 12. The computer-implemented method of claim 11, configured to:

13. 11. The computer-implemented method of claim 10, further comprising using the stored usage data for one or more sensitivity analyses.

14. checking one or more credentials of the user before performing the one or more operations on the D / E data; determining that the user is entitled to perform the one or more operations on the D / E data based on the one or more credentials; 10. The computer-implemented method of claim 1, further comprising:

15. 2. The computer-implemented method of claim 1, wherein the step of receiving the D / E data from the user device includes receiving a request from a customer environment at a digital engineering platform enclave, the customer environment and the digital engineering platform enclave being managed by different entities, and the request from the customer environment cannot modify production software associated with the digital engineering platform enclave.

16. a memory for storing executable instructions; one or more processing devices coupled to the memory, receiving design and / or engineering data (D / E data) from a user device corresponding to a prototype representation of the product; transmitting one or more inputs derived from said D / E data to one or more digital engineering tools for processing; receiving engineering-related data output from the one or more digital engineering tools; receiving data corresponding to one or more common validation and verification (V&V) products associated with said product; identifying one or more requirements for the one or more common V&V products based on the data corresponding to the one or more common V&V products; determining whether the one or more requirements have been met based on the engineering-related data output and the data corresponding to the one or more common V&V products; presenting at the user device information corresponding to the engineering-related data output and / or the data corresponding to the one or more common V&V products, the presented information including an indication of whether the one or more requirements have been met or an indication of a probability of whether the one or more requirements are met by the prototype representation of the product; receiving instructions from the user device after presenting the information corresponding to the engineering-related data output and / or the data corresponding to the one or more common V&V products at the user device, the instructions corresponding to one or more user interactions with the user device; and performing one or more operations on the D / E data in response to receiving the instruction from the user device. one or more processing devices configured to execute the instructions to perform operations including Including, the system.

17. 17. The system of claim 16, wherein the data corresponding to the one or more common V&V products is received from the user device.

18. 17. The system of claim 16, wherein the data corresponding to the one or more common V&V products is received from a regulatory and / or certification authority.

19. The system of claim 16 , wherein the product is an aircraft.

20. 17. The system of claim 16, wherein at least a subset of the one or more digital engineering tools comprises a model-based systems engineering (MBSE) tool, an augmented reality (AR) tool, a computer-aided design (CAD) tool, a data analytics tool, a modeling and simulation (M&S) tool, a product lifecycle management (PLM) tool, a simulation engine, a requirements model, an electronics model, a test planning model, a cost model, a schedule model, a software modeling, a supply chain model, a manufacturing model, a cybersecurity model, a multi-attribute tradespace tool, or a mission effects model.

21. 17. The system of claim 16, wherein determining whether the one or more requirements have been met based on the engineering-related data output comprises determining whether the one or more requirements have been met without any human input.

22. The system of claim 16 , wherein the presented information further includes recommended actions that the user of the user device can take to satisfy the one or more requirements.

23. 23. The system of claim 22, wherein the recommended action comprises a suggestion to use a particular digital engineering tool of the one or more digital engineering tools, a suggestion to modify the one or more inputs sent to the one or more digital engineering tools, a suggestion to modify one or more components of the prototype representation of the product, and / or a suggestion to replace one or more components of the prototype representation of the product with a previously designed solution.

24. 17. The system of claim 16, wherein performing the one or more operations on the D / E data includes modifying the D / E data and / or deriving modified inputs from the D / E data for transmission to the one or more digital engineering tools.

25. 17. The system of claim 16, wherein the operations further include storing in a storage device usage data representative of the received data corresponding to the one or more common V&V products, the received D / E data, the engineering-related data output from the one or more digital engineering tools, the indication of whether the one or more requirements have been met, the indication of the probability of whether the one or more requirements are met by the prototype representation of the product, the one or more interactions of the user with the user device, and / or the one or more operations performed on the D / E data.

26. The operation is incorporating at least a portion of the usage data into a training data set; training a machine learning model based on the training dataset; 26. The system of claim 25, further comprising:

27. The machine learning model: receiving as input information about another product being designed by another user; outputting a proposal for the other user to use a particular one of the one or more digital engineering tools, a proposal to modify one or more inputs submitted by the other user to the one or more digital engineering tools, a proposal to modify one or more components of another prototype representation associated with the other user, a proposal to replace one or more components of the other prototype representation with a previously designed solution, and / or a proposal for a completely or partially new design generated using a machine learning engine; 27. The system of claim 26, configured to:

28. 26. The system of claim 25, further comprising using the stored usage data for one or more sensitivity analyses.

29. The operation is checking one or more credentials of the user before performing the one or more operations on the D / E data; determining that the user is entitled to perform the one or more operations on the D / E data based on the one or more credentials; 17. The system of claim 16, further comprising:

30. receiving design and / or engineering data (D / E data) from a user device corresponding to a prototype representation of the product; transmitting one or more inputs derived from said D / E data to one or more digital engineering tools for processing; receiving engineering-related data output from the one or more digital engineering tools; receiving data corresponding to one or more common validation and verification (V&V) products associated with said product; identifying one or more requirements for the one or more common V&V products based on the data corresponding to the one or more common V&V products; determining whether the one or more requirements have been met based on the engineering-related data output and the data corresponding to the one or more common V&V products; presenting at the user device information corresponding to the engineering-related data output and / or the data corresponding to the one or more common V&V products, the presented information including an indication of whether the one or more requirements have been met or an indication of a probability of whether the one or more requirements are met by the prototype representation of the product; receiving instructions from the user device after presenting the information corresponding to the engineering-related data output and / or the data corresponding to the one or more common V&V products at the user device, the instructions corresponding to one or more user interactions with the user device; and performing one or more operations on the D / E data in response to receiving the instruction from the user device. [0023] One or more non-transitory machine-readable storage media storing instructions that are executed to perform operations including:

Images