Digital threads for defining software code in digital engineering systems using artificial intelligence (AI)

Abstract

A method and system are provided for generating digital threads that connect models and software tools for an interconnected digital engineering and certification ecosystem. In one embodiment, the process for generating a software code-defined digital thread includes receiving a first model representation of a first engineering model, a second model representation of a second engineering model, and intent input; using a machine learning (ML) model to determine one or more model endpoints in the first and second model representations; using a script generation ML model to generate a platform orchestration script that performs the intent input by connecting the first and second model representations using one or more model endpoints; and finally, storing the platform orchestration script as a software code-defined digital thread. With or without AI assistance, the digitalization of digital engineering models exponentially improves the efficiency of designing, testing, certification, manufacturing, and operating physical products, enabling the manufacturing of physical products at digital speeds.

Description

Technical Field

[0001] Reference to Related Applications If an Application Data Sheet ("ADS") or PCT Request Form ("Request") has been filed on the filing date of this application, it shall be incorporated herein by reference. Any application claimed in the ADS or Request for priority under 35 U.S.C. §§ 119, 120, 121, or 365(c), and any and all parent applications, grandparent applications, great-grandparent applications, etc. of such application, as well as any priority claims made in those applications and any materials incorporated by reference, shall also be incorporated herein by reference, to the extent that their subject matter is not inconsistent with this specification.

[0002] Furthermore, this application is related to the following U.S. patent applications, which are hereby incorporated by reference in their entirety as if fully set forth herein. ● PCT Patent Application No. PCT / US24 / 18278 (Docket No. IST-02.001PCT), entitled "Secure and Scalable Model Splicing of Digital Engineering Models for Software-Code-Defined Digital Threads," filed on March 3, 2024, describes model splicing for digital engineering platforms. ● PCT Patent Application No. PCT / US24 / 14030 (Docket No. IST-01.001PCT), entitled "Artificial Intelligence (AI) Assisted Digital Documentation for Digital Engineering," filed on February 1, 2024, describes AI-assisted documentation for digital engineering platforms. ● U.S. Provisional Patent Application No. 63 / 442,659 (IST-01.001P), filed on February 1, 2023, entitled "AI-Assisted Digital Documentation for Digital Engineering with Supporting Systems and Methods," describes AI-assisted tools for modeling and simulation applications of digitally designed products, as well as for engineering (DE), including certification. ● U.S. Provisional Patent Application No. 63 / 451,545 (IST-01.002P), filed on March 10, 2023, entitled "Digital Threads in Digital Engineering Systems, and Supporting AI-Assisted Digital Thread Generation," describes a model splicer and digital threading technology. ● U.S. Provisional Patent Application No. 63 / 451,577 (IST-02.001P1), filed on March 11, 2023, entitled "Model Splicer and Microservice Architecture for Digital Engineering," describes model splicer technology. ● U.S. Provisional Patent Application No. 63 / 462,988 (IST-02.001P2), filed on April 29, 2023, entitled "Model Splicer and Microservice Architecture for Digital Engineering," describes model splicer technology. ● U.S. Provisional Patent Application No. 63 / 511,583 (Reference Number IST-02.002P), filed on June 30, 2023, entitled "AI-Assisted Model Splicer Generation for Digital Engineering," describes AI-assisted model splicer technology. ● U.S. Provisional Patent Application No. 63 / 516,624 (IST-02.003P), filed on July 31, 2023, entitled "Document and Model Splicing for Digital Engineering," describes document splicing technology. ● U.S. Provisional Patent Application No. 63 / 520,643 (Reference Number IST-02.004P), filed on August 20, 2023, entitled "Artificial Intelligence (AI)-Assisted Automation of Testing in a Software Environment," describes software testing using AI assistance. ● U.S. Provisional Patent Application No. 63 / 590,420 (IST-02.005P), filed on October 14, 2023, entitled "Commenting and Collaboration Capability within Digital Engineering Platform," describes collaborative capabilities. ● U.S. Provisional Patent Application No. 63 / 586,384 (IST-02.006P), filed on September 28, 2023, entitled "Artificial Intelligence (AI)-Assisted Streamlined Model Splice Generation, Unit Testing, and Documentation," describes AI-assisted streamlined model splicing, testing, and documentation. ● U.S. Provisional Patent Application No. 63 / 470,870 (IST-03.001P), filed on June 3, 2023, entitled "Digital Twin and Physical Twin Management with Integrated External Feedback within a Digital Engineering Platform," describes the management of digital and physical twins and the integration of external feedback within a DE platform. ● U.S. Provisional Patent Application No. 63 / 515,071 (IST-03.002P), filed on July 21, 2023, entitled "Generative Artificial Intelligence (AI) for Digital Engineering," describes an AI-enabled digital engineering task execution process within a DE software platform. ● U.S. Provisional Patent Application No. 63 / 517,136 (IST-03.003P), filed on August 2, 2023, entitled "Machine Learning Engine for Workflow Enhancement in Digital Engineering," describes a machine learning engine for model splicing and DE script generation. ● U.S. Provisional Patent Application No. 63 / 516,891 (IST-03.004P), filed on August 1, 2023, entitled "Multimodal User Interfaces for Digital Engineering," describes a multimodal user interface for DE systems. ● U.S. Provisional Patent Application No. 63 / 580,384 (IST-03.006P), filed on September 3, 2023, entitled "Multimodal Digital Engineering Document Interfaces for Certification and Security Reviews," describes a multimodal user interface for certification and security reviews. ● U.S. Provisional Patent Application No. 63 / 613,556 (IST-03.008P), filed on December 21, 2023, entitled "Alternative Tool Selection and Optimization in an Integrated Digital Engineering Platform," describes tool selection and optimization. ● U.S. Provisional Patent Application No. 63 / 584,165 (IST-03.010P), filed on September 20, 2023, entitled "Methods and Systems for Improving Workflows in Digital Engineering," describes workflow optimization in a DE platform. ● U.S. Provisional Patent Application No. 63 / 590,456 (IST-04.001P), filed on October 15, 2023, entitled "Data Sovereignty Assurance for Artificial Intelligence (AI) Models," relates to the assurance of data sovereignty during AI model training and evaluation. ● U.S. Provisional Patent Application No. 63 / 606,030 (IST-04.001P2), filed on December 4, 2023, entitled "Data Sovereignty Assurance for Artificial Intelligence (AI) Models," provides further details on guaranteeing data sovereignty during AI model training and evaluation. ● U.S. Provisional Patent Application No. 63 / 419,051 (Reference Number 54332-0059P01) titled "Interconnected Digital Engineering and Certification Ecosystem," filed on October 25, 2022. ● U.S. Non-Provisional Patent Application No. 17 / 973,142 (Reference Number 54332-0057001) titled "Interconnected Digital Engineering and Certification Ecosystem," filed on October 25, 2022. ● U.S. Non-Provisional Patent Application No. 18 / 383,635 (Reference Number 54332-0059001) titled "Interconnected Digital Engineering and Certification Ecosystem," filed on October 25, 2023. ● U.S. Provisional Patent Application No. 63 / 489,401 (Reference Number 54332-0063P01) titled "Security Architecture for Interconnected Digital Engineering and Certification Ecosystem" was filed on March 9, 2023.

[0003] Copyright and Trade Dress Notice Some disclosures in this patent document include copyrighted material. This patent document may indicate and / or describe matters that are or may become the rights holder's trade dress. The copyright holder and the rights holder of the trade dress will not object to any reproduction by any person of the patent disclosure as it appears in the U.S. Patent and Trademark Office application or record, but reserves all other copyright and trade dress rights.

[0004] ISTARI DIGITAL is a trademark name that conveys embodiments of the present invention, and therefore the aforementioned trademark name may be used interchangeably in the specification and drawings to refer to the products / processes provided by embodiments of the present invention. The terms ISTARI and ISTARI DIGITAL may be used herein to describe the present invention and the company providing the invention.

[0005] This disclosure relates to software tools for digital engineering, including modeling and simulation applications, and for the certification of digitally designed products. Specifically, this disclosure relates to methods and systems for creating, managing, and executing digital threads that connect engineering models and associated software tools within such ecosystems. [Background technology]

[0006] The background information provided in this invention is intended to aid in understanding the invention and its applications and uses, and may not constitute prior art.

[0007] Digital engineering software tools, including modeling and simulation tools that accurately virtualize physical systems or processes in relation to real-world decisions, enable the iterative and effective development of systems and components. To enable digital engineering from the design of complex systems to validation, verification, and certification, heterogeneous engineering tools from multiple disciplines are required. These digital engineering tools and the engineering models they generate are siloed across different software tools. Integrating data and models from siloed tools is one of the most expensive aspects of digital engineering, requiring a large team of highly specialized engineers and software developers. Furthermore, because digital engineering tools are constantly evolving, the integration work is never-ending. Therefore, integrating heterogeneous models or model type files requires ongoing maintenance by a large team of engineers and software developers, including highly expensive subject matter specialists.

[0008] Furthermore, the certification of these systems and components is complex, requiring the integration of data from engineering models from heterogeneous tools, along with human-readable documentation, throughout the entire certification process. Moreover, certification still requires information and testing primarily occurring in the physical world, using the physical manifestation of digitally designed systems and components (often referred to herein as “products”). Additionally, physical tests completed for other work or by other third-party stakeholders (e.g., component suppliers) are often repeated. This is because third-party stakeholders may not want to share complete data from previous tests. This results in redundant physical testing, adding cost and delays to the development and certification work.

[0009] Therefore, enabling the integration of multidisciplinary engineering models from disparate software tools, along with human-readable documentation, in an interconnected digital engineering platform would represent a technological advancement. Interconnected digital engineering platforms enable the design, validation, verification, and certification of complex systems. For example, such a platform could be used to ensure the accuracy and reliability of validation and certification processes required to obtain full digital certification of a new aerospace vehicle, a new automobile, or even a new biomedical device or chemical process, thereby reducing or completely eliminating the need for pre-certification physical testing.

[0010] Based on this background technology, the present invention was developed. [Overview of the project]

[0011] This summary of the invention provides a general overview of the present invention, its applications, and use, and is not intended to limit the scope of the invention, which will become clear from the detailed description when read in conjunction with the drawings.

[0012] One embodiment of the present invention is an interconnected digital engineering platform that enables the generation of software code definition digital threads, with or without AI assistance. The software code definition digital thread links two or more engineering models from heterogeneous software tools. In some embodiments, one of the engineering models may be a human-readable document. In some embodiments, the software code definition digital thread links one or more other data sources (e.g., live test data).

[0013] Therefore, various methods, systems, and non-temporary storage media for storing program code for executing processes for generating software core definition digital threads in digital engineering systems are within the scope of the present invention.

[0014] According to the first embodiment, a non-temporary physical storage medium for storing program code is provided. The program code is executable by a hardware processor. When the hardware processor executes the program code, it causes the hardware processor to execute a computer execution process for generating a software code definition digital thread. The program code includes code that can train a script-generating machine learning (ML) model using a training dataset which includes a set of training triplets each containing a sample intent input, a corresponding set of sample model representations, and a corresponding sample platform orchestration script, the sample platform orchestration script connecting the models in the corresponding set of sample model representations to perform the corresponding sample intent input. The program code may include code for receiving a first model representation of a first engineering model. The program code may include code for receiving a second model representation of a second engineering model. The program code may include code for receiving intent input. The program code may include code for generating a platform orchestration script that connects the first and second model representations based on the intent input using the script-generating ML model. The platform orchestration script can perform the intent input. The program code may include code for storing platform orchestration scripts as software code-defined digital threads.

[0015] In one embodiment, the non-temporary storage medium further includes program code for receiving feedback data relating to the platform orchestration script. The program code may include code for training and / or fine-tuning the script generation ML model based on the feedback data.

[0016] In one embodiment, the non-temporary storage medium further includes program code to provide a user interface coding environment in an interconnected digital engineering platform (IDEP). The program code may include code for receiving multiple user selections of a first engineering model and a second engineering model. The first and second engineering models may be selected by the user. The program code may include code for receiving multiple corresponding model representations from the first and second engineering models. The program code may include code for receiving user-defined code for a user-defined platform orchestration script. The program code may include code for determining and / or receiving corresponding intent inputs. The program code may include code for determining corresponding model representation endpoints used in the user-defined code from the user-defined platform orchestration script. The program code may include code for recording the first and second engineering models, the first and second model representations, corresponding intent inputs, and corresponding model representation endpoints, as well as a user-defined platform orchestration script for generating a training dataset. The program code may include code for storing a training dataset for training a script-generated ML model.

[0017] In some embodiments, connecting a first model representation and a second model representation based on intent input includes linking a first endpoint of the first model representation and a second endpoint of the second model representation based on intent input.

[0018] In one embodiment, the non - transient storage medium further includes program code for evaluating a first engineering model and a second engineering model within an Interconnected Digital Engineering Platform (IDEP) for sufficiency to perform an intent input using a sufficient machine learning (ML) model.

[0019] In one embodiment, the non - transient storage medium further includes program code for determining a first endpoint in a first model representation related to an intent input using a recommender ML model or a script generation machine learning (ML) model in response to a determination that sufficiency has been determined.

[0020] In one embodiment, the non - transient storage medium further includes program code for determining a relationship between a first endpoint and a second endpoint based on an intent input using a recommender ML model.

[0021] In some embodiments, the platform orchestration script includes script code for reading data from a first model representation and / or a second model representation.

[0022] In some embodiments, the platform orchestration script includes script code for writing data to a first model representation and / or a second model representation.

[0023] In some embodiments, the platform orchestration script includes an input for a second model representation connected to the output of a first model representation.

[0024] In one embodiment, the non - transient storage medium further includes program code for executing a platform orchestration script for a second model representation. The output from the first model representation can be an input for the second model representation.

[0025] In one embodiment, the non-temporary storage medium further includes code for reading data from the first model representation. The program code may include code for performing calculations on the data. The program code may include code for writing the results of the calculations to the first and / or second model representations.

[0026] In one embodiment, the non-temporary storage medium further includes program code for receiving a third model representation of the third engineering model. The platform orchestration script may further link the first and / or second model representations with the third model representation.

[0027] In one embodiment, the non-temporary storage medium further includes code for executing a platform orchestration script by calling one or more API endpoints or SDK endpoints associated with a first model representation and / or a second model representation.

[0028] In one embodiment, the non-temporary storage medium further includes program code for using an AI model to determine a recommended third engineering model based on a first engineering model, a second engineering model, and a training dataset.

[0029] In some embodiments, the first engineering model and / or the second engineering model is a human-readable document file.

[0030] In one embodiment, the non-temporary storage medium further includes program code for receiving a document template. The program code may include code for analyzing the document template using an interconnected digital engineering platform (IDEP). The program code may include code for using an AI model to determine the output data from a first model representation and / or a second model representation required to generate a document file. The program code may include code for performing appropriate actions on the first model representation and / or the second model representation using a predetermined sequence based on the requirements of the document template in order to generate the output required for the document file. The program code may include code for generating the document file by assembling the document template and the output from the first model representation and / or the second model representation.

[0031] In one embodiment, the non-temporary storage medium further includes program code for predicting changes in the first model representation of the first engineering model and / or the second model representation of the second engineering model based on the changes in the first model representation of the first engineering model and / or the second model representation of the second engineering model.

[0032] In one embodiment, the non-temporary storage medium further includes program code for predicting changes in a first model representation of a first engineering model based on changes in a second model representation of a second engineering model.

[0033] In one embodiment, the non-temporary storage medium further includes program code for calling a second software code-defined digital thread.

[0034] In some embodiments, the first engineering model and / or the second engineering model include a neural network model.

[0035] In one embodiment, the non-temporary storage medium further includes program code for generating a magic document associated with a software code definition digital thread using an AI model. The magic document may include an API endpoint to a human-readable text block. The magic document may be updated in the audit log using the API endpoint in response to the execution of at least a portion of the platform orchestration script.

[0036] In some embodiments, the platform orchestration script includes a code block. The code block may be associated with an information security tag. The information security tag may indicate restrictions on the execution of the code block.

[0037] In one embodiment, the model representation may be a model splice, and the non-temporary storage medium further includes program code for receiving a first engineering model file of a first engineering model having a DE model type. The first engineering model file may be in a native file format. The program code may include code for extracting model data from the first engineering model file in the native file format. The program code may include code for storing the model data in a model data storage area. The program code may include code for generating one or more externally accessible splice functions that enable external access to one or more digital artifacts derived from the model data stored in the model data storage area. One or more externally accessible splice functions provide addressable application programming interface (API) endpoints or software development kit (SDK) endpoints that may be accessible by third-party applications and users. The API or SDK endpoints may enable access to the digital artifacts without accessing the entire first engineering model file and without requiring direct involvement by third-party applications and users using DE tools associated with the DE model type. The program code may include code for generating a first model splice of the first engineering model. The first model splice may include access to a selection of one or more digital artifacts. The first model splice may include access to at least one of one or more externally accessible splice functions. The first model splice may be accessible by third-party applications and users via an API endpoint or SDK endpoint. The API endpoint or SDK endpoint may provide a unified programming interface to shareable model splices generated from DE models having DE model types.

[0038] According to a second aspect, a computer implementation method for generating a software code definition digital thread is provided. The method may include the following steps: training a script-generating machine learning (ML) model using a training dataset including a set of training triplets, each containing a sample intent input, a corresponding set of sample model representations, and a corresponding sample platform orchestration script; the sample platform orchestration script may connect the corresponding set of sample model representations to perform the corresponding sample intent input; receiving a first model representation of a first engineering model, receiving a second model representation of a second engineering model, and receiving intent input; generating a platform orchestration script using the script-generating ML model that connects the first and second model representations based on the intent input; the platform orchestration script may perform the intent input; and finally, storing the platform orchestration script as a software code definition digital thread.

[0039] In another embodiment of the present invention, a non-temporary computer-readable storage medium is provided which, when executed by a processor, stores executable instructions causing the processor to perform a process for generating a digital thread, including the steps described above.

[0040] In yet another aspect or embodiment of the present invention, a computer program product is provided. The computer program may be used for generating digital threads and may include a computer-readable storage medium in which program instructions or program code are embodied, and the program instructions are executable by a processor to cause the processor to perform the steps described above.

[0041] In another aspect or embodiment of the present invention, a system for generating digital threads is provided, the system comprising a memory for storing computer executable components and a hardware processor operably coupled to the memory and for executing the computer executable components stored in the memory, the computer executable components may include components operably coupled to the processor that performs the steps described above.

[0042] In yet another aspect or embodiment of the present invention, a system for generating digital threads is provided, the system comprising a user device having a processor, a display, and a first memory; a server having a second memory and a data repository; a communication link between the user device and the server; and a plurality of computer codes embodied on the first memory and the second memory of the user device and the server, the plurality of computer codes, when executed, cause the server and the user device to perform a process including the steps described herein.

[0043] In another aspect or embodiment of the present invention, a computerized server is provided which includes at least one processor, memory, and a plurality of computer codes embodied on the memory, wherein, when executed, the plurality of computer codes cause the processor to execute a process including the steps described herein. Other aspects and embodiments of the present invention include methods, processes, and algorithms including the steps described herein, and also include processes and operating modes of the systems and servers described herein.

[0044] In yet another aspect or embodiment of the present invention, an edge computerization system is provided, which is executed on a physical system or physical twin (PTw) using either processing, memory, computer code stored on a non-temporary computer-readable storage medium of the physical system or PTw, and access to a plurality of sensor data measured on the physical system or PTw, or dedicated processing, dedicated memory, dedicated computer code stored on a non-temporary computer-readable storage medium of the physical system or PTw, and a plurality of dedicated sensor data measured on the physical system or PTw, wherein the computer code causes a processor to perform the steps described above.

[0045] Features described in the context of different aspects and / or embodiments of the present invention may be used together and / or interchangeable wherever possible. Similarly, where features are described in the context of a single embodiment for the sake of brevity, those features may also be provided separately or in any suitable partial combination. Features described in relation to non-temporary physical storage media may have corresponding features that are definable and / or combinatable with respect to digital documentation systems and / or methods and / or systems, and vice versa, as these embodiments specifically envision.

[0046] Further other aspects and embodiments of the present invention will become apparent from the modes for carrying out the invention when read in conjunction with the accompanying drawings.

[0047] The accompanying drawings are incorporated into and constitute part of this specification, illustrating embodiments of the invention and are used in conjunction with this description to illustrate the principles of the disclosed embodiments. For clarity, brevity, and flexibility, not all elements, components, or specifications are defined in all drawings. Not all drawings corresponding to specific steps or embodiments of the invention are drawn to scale. Instead, emphasis is placed on the description of the properties, functions, and products of the manufacturing methods and devices described herein.

[0048] The embodiments of the present invention described herein are illustrative and not limiting. Hereinafter, embodiments will be described by reference to the accompanying drawings, and section headings are provided for the sake of clarity. [Brief explanation of the drawing]

[0049] Interconnected Digital Engineering Platform [Figure 1] The following describes exemplary interconnected digital engineering platform (IDEP) architectures according to several embodiments of the present invention. [Figure 2] This document illustrates exemplary embodiments of IDEP as an interconnected digital engineering (DE) and certification ecosystem, as well as exemplary digitally certified products, according to several embodiments of the present invention. [Figure 3] This document presents another exemplary embodiment of IDEP, illustrating the services and features provided by some embodiments of the present invention. [Figure 4] This invention presents potential scenarios for instantiating an IDEP connected to a customer's physical system and IT environment, according to several embodiments of the present invention. [Figure 5] This document illustrates exemplary multimodal interface designs for feedback integration in IDEP according to several embodiments of the present invention. The digital engineering platform links digital models to digital threads. [Figure 6] This is a schematic diagram comparing exemplary digital threads connecting DE models according to several embodiments of the present invention. [Figure 7] This is a schematic diagram illustrating an exemplary DE model splicing setup according to several embodiments of the present invention. [Figure 8] This is a schematic diagram illustrating the digital threading of a DE model by model splicing according to several embodiments of the present invention. [Figure 9] This is a schematic diagram illustrating the linking of DE model splices in a splice plane according to several embodiments of the present invention, and a comparison between digital threading with model splicing and digital threading without model splicing. [Figure 10] This document presents exemplary directed acyclic graph (DAG) representations of pipelined DE tasks related to digital threads according to several embodiments of the present invention. It also provides an AI-assisted, versatile link for generating digital threads. [Figure 11] An illustrative schematic diagram is shown of data from a digital thread used to train an AI algorithm to support a user's workflow, according to several embodiments of the present invention. [Figure 12] This diagram illustrates an exemplary AI-assisted digital thread enabling various DE services according to several embodiments of the present invention. [Figure 13] The following describes a process flow for generating a software code definition digital thread according to several embodiments of the present invention. [Figure 14] This document outlines the IDEP neural network training process according to several embodiments of the present invention. [Figure 15] This diagram shows an illustrative schematic of a digital engineering tool applied to requirements files and design files according to several embodiments of the present invention. [Figure 16] Examples of steps for implementing scalable model sharing according to several embodiments of the present invention are shown. [Figure 17]This diagram shows an exemplary schematic representation of AI-assisted multipurpose linking of MBSE files according to several embodiments of the present invention. [Figure 18] This document illustrates an exemplary process for extracting DE model (CAD or FEA) data for sharing, according to several embodiments of the present invention. [Figure 19] The present invention illustrates an exemplary process for generating Magic Dock-type documentation according to several embodiments of the present invention. [Figure 20] This illustrates the linking of CAD and FEA models 1815 and analysis documentation 1915 according to an exemplary embodiment of the present invention. It also shows the generation and updating of digital threads and associated magic documents. [Figure 21] The swimlanes of the update process flow for the digital thread and associated magic document according to several embodiments of the present invention are shown. [Figure 22] The first part of a detailed process flow for recommending, creating, and updating digital threads and magic documents according to several embodiments of the present invention is shown. [Figure 23] A second part of a detailed process flow for recommending, creating, and updating digital threads and magic documents according to several embodiments of the present invention is shown. [Figure 24] This specification provides a detailed process flow for creating digital threads and associated magic documents using a generative AI-assisted approach, as illustrated by the examples disclosed herein. Exemplary Digital Thread Graphical User Interface [Figure 25] This specification provides a graphical user interface (GUI) associated with an exemplary process flow for verifying and authenticating requirements within an IDEP, as disclosed herein. [Figure 26] The image shows a screenshot of an exemplary graphical user interface (GUI) used to manipulate digital threads on an IDEP, according to one embodiment of the present invention. [Figure 27]The image shows a screenshot of another exemplary graphical user interface (GUI) used to manipulate digital threads on IDEP, according to one embodiment of the present invention. [Figure 28] A screenshot of an exemplary graphical user interface (GUI) used with a digital documentation system according to one embodiment of the present invention is shown. Machine learning implementation architecture for AI-assisted digital threads. [Figure 29] The fundamentals of neural network operation according to several embodiments of the present invention will be described. [Figure 30] This document outlines the IDEP neural network training process according to several embodiments of the present invention. [Figure 31] This is an exemplary flowchart illustrating different phases and datasets involved in training an IDEP machine learning model according to several embodiments of the present invention. Hardware and software architecture for AI-assisted digital threads. [Figure 32] This invention provides illustrative schematic diagrams of a server (management computing entity) and a client (user computing entity) used for documentation within an IDEP, according to several embodiments of the present invention. [Modes for carrying out the invention]

[0050] In the following description, many specific details are given for illustrative purposes to provide a complete understanding of the invention. However, it will be apparent to those skilled in the art that the invention can be implemented without using these specific details. In other examples, structures, devices, activities, methods, and processes are shown using schematic diagrams, use cases, and / or illustrations to avoid obscuring the invention. The following description contains many details for illustrative purposes, but those skilled in the art will understand that many variations and / or modifications to the details presented fall within the scope of the invention. Similarly, many of the features of the invention are described in relation to or in relation to each other, but those skilled in the art will understand that many of these features can be provided independently of others. Accordingly, this description of the invention is written without prejudice to the generality of the invention and without imposing limitations on the invention.

[0051] Digital transformation represents a rapidly expanding market characterized by robust profit margins. However, its growth is hampered by unique challenges. Specifically, the creation of digital twins through interconnected models and simulations, known as "digital threads," is hindered by issues such as vendor lock-in, high licensing costs, and technical debt. The technology landscape for digital transformation is well-funded and dynamic, encompassing a range of technologies from the Internet of Things (IoT) and cloud-to-edge computing to API-first and code-first hardware, as well as advanced large-scale language models and AI. This presents substantial opportunities to integrate models and simulations with these technologies, thereby simplifying the creation of digital threads. Digital twins are part of the broader concept of Industry 4.0, envisioned as interconnected models that not only simulate but also enhance our physical reality. Despite their potential, over 90% of digital transformation initiatives struggle to achieve success. When successful, they bring unparalleled capabilities for optimization and innovation to companies, particularly through the use of AI in complex industrial systems. This invention envisions a future where technological barriers to digital transformation are eliminated, and digitalization becomes an easily accessible commodity. By integrating models and simulations through digital threads, this invention envisions the creation of an industrial metaverse, the democratization of technological innovation, and the provision of rich data for AI to learn. Instead of the current labor-intensive process of creating digital threads, this invention enables their mass production. This includes a tunable application layer between models and simulations uploaded to an Interconnected Digital Engineering Platform (IDEP), thereby facilitating customization and maintaining ease of integration even as models evolve. This invention enables the mass production of digital threads.By building an application layer on top of the model splice, the platform provides ease of customization and integration, as well as a complete suite of applications from third-party developers. The ultimate goal is to create a digital engineering ecosystem similar to a software development stack, delivering significant customer value and a positive user experience.

[0052] Embodiments of the present invention will now be described in detail with reference to the drawings. First, general digital engineering (DE) systems and terminology will be introduced. Next, an interconnected digital engineering platform (IDEP) will be described in detail. Finally, a digital threading system, which may be considered a subsystem of the IDEP with or without AI assistance, will be described in detail.

[0053] General terminology To aid in understanding the present invention, several illustrative terms used with IDEP are given below, but these should not be read as limiting the scope of the invention. Terms may be used in noun, verb, or adjective form within the scope of their definition. ● Digital Engineering (DE): According to the Defense Acquisition College (DAU) and the U.S. Department of Defense (DOD) Digital Engineering Strategy published in 2018, digital engineering is "an integrated digital approach to systems engineering that supports lifecycle activities from concept to disposal using trusted sources and models of system data as a cross-disciplinary continuum." Digital engineering reinforces the paradigm shift in systems engineering from traditional design-build-test methodologies to new model-analysis-build methodologies, thereby incorporating digital innovations into an integrated model-based approach that enables system design, prototyping, and testing all within a virtual environment. ● DE Data: Digital Engineering (DE) data includes project management, program management, product management, design review, and / or engineering data. ● DE Data Field: For example, a data field for DE data within a DE document template. ● Phases: Stages within the DE product lifecycle, including but not limited to stakeholder analysis, conceptual studies, requirements definition, preliminary design and technical review, system modeling, final design, implementation, system assembly and integration, prototyping, verification and validation at the system, subsystem, and component levels, and operation and maintenance. ● DE Model, also called a "Digital Model": A computer-generated model that represents the characteristics or behavior of a complex product or system. A DE model can be created or modified using DE tools, and a DE model may be represented by one or more DE model files. A DE model file is a computer model file created or modified using DE tools. In this disclosure, the terms “Digital Model,” “DE Model,” and “DE Model File” may be used interchangeably where the context requires. A DE Model in IDEP disclosed herein refers to any digital file uploaded to the platform, including documents that are appropriately interpreted as defined below. For example, in various embodiments of the invention, a computer-aided design (CAD) file, a system modeling language (SysML) file, a system requirements definition (SDR) text file, and a neural network model JSON file may each be considered a DE model. A DE model may be machine-readable only, human-readable but written in programming code, or human-readable but written in natural language-based text. For example, a document processing document containing a product's technical specifications, or a spreadsheet file containing technical data about a product, may also be considered a DE model. ● Interconnected Digital Engineering Platform (IDEP), also known as a “Digital Engineering and Certification Ecosystem”: According to the DAU, a “DE Ecosystem” is “an interconnected infrastructure, environment, and methodology (processes, methods, and tools) used to store, access, analyze, and visualize data and models of evolving systems to address the needs of stakeholders.” Embodiments of IDEP disclosed herein include a software platform operating on hardware to achieve the aforementioned capabilities under zero-trust principles. DE and certification ecosystems perform verification and validation tasks as defined below. ● Verification: According to DAU, verification "confirms that system elements meet design or construction specifications. Throughout the system lifecycle, design solutions at all levels of the physical architecture are verified by a cost-effective combination of analysis, inspection, demonstration, and testing." Verification refers to evaluating whether a product, service, or system meets specified requirements and is suitable for its intended purpose, and to externally check the needs of customers or stakeholders. For example, in the aviation industry, the verification process may include testing aircraft components to ensure that they can withstand the forces and conditions encountered during flight. ● Validation: According to the DAU, validation is "1) the review and approval of capability requirements documents by a designated validation body, 2) the process by which the contractor tests publications / technical manuals for technical accuracy and validity (or in a manner directed by the DoD component procurement activity), and 3) the process of evaluating a system or software component during or at the end of the development process to determine whether it meets the specified requirements." Therefore, validation refers to evaluating whether the overall performance of a product, service, or system is suitable for its intended use, including compliance with regulatory requirements, and its ability to meet the needs of its intended users, and internally checking against specifications and regulations. For example, in the manufacturing of industrial products, the validation process may include consumer surveys to inform the product design, modeling and simulation to validate the design, prototype testing for failure limits, and surveys of feedback from buyers. ● Common Verification and Validation (V&V) Products: Regulatory and certification standards, compliance, calculations, and tests (e.g., for the development, testing, and certification of products and / or solutions) are referred to herein as “Common V&V Products.” ● DE Tool: A tool or DE tool is a DE application software (e.g., CAD software), a computer program, and / or a script that creates or manipulates a DE model during at least one stage or phase of the product lifecycle. A DE tool may include multiple functions or methods. ● Application Programming Interface (API): A software interface that provides software programs with programmatic access to services, enabling application software to exchange data and communicate with each other using standardized requests and responses. This allows different programs to work together without revealing the internal details of how each program operates. DE tools typically provide API libraries for code interface access. ● Script: A sequence of instructions that are interpreted and executed within another program, or executed by another program, without being compiled into a binary file that can be executed on its own through a computer processor without the support of other programs. ● API Script: A script that implements specific functionality available through IDEP, such as those disclosed herein. An API script may be an API function script encapsulated in a model splice, or an "orchestration script" or "platform script" that orchestrates a workflow through digital threads built on interconnected model splices. ● Platform API or ISTARI API: A library of API scripts available in IDEP, such as those disclosed herein. ● API function scripts, "splice functions," "splice methods," "ISTARI functions," or "function nodes": A type of API script. When executed, API function scripts input to or output from DE models or DE model splices. "Input" functions, input methods, or "input nodes" allow updating or modifying the input DE model. "Output" functions, output methods, or "output nodes" allow data extraction or derivation from the input DE model via its model splice. API function scripts may also invoke native API function calls of native DE tools, and the terms "native" and "primal" may refer to existing DE model files, functions, and API libraries associated with a particular third-party DE tool, including proprietary and open-source ones. ● Endpoint: In the context of software and networking, an endpoint is a specific digital location or destination from which different software systems communicate with each other. This allows external systems to access the functionality or data of an application, operating system, or other service. An API endpoint is a point of interaction where an API receives a request and returns data in response. Software Development Kit (SDK) endpoints or SDK-defined endpoints similarly provide service handles for use with the SDK. References to API endpoints in this disclosure are equally applicable to SDK endpoints. ● Deliverables: According to DAU, digital deliverables are “deliverables created within or generated from the DE ecosystem” for “providing data for alternative views, to visualize, communicate, and deliver data, information, and knowledge to stakeholders.” In this disclosure, “digital deliverables” or “deliverables” are the results of execution from output API function scripts within a model splice. Multiple deliverables may be generated from a single DE model or DE model splice. ● Model Splice: Within this disclosure, a “model splice,” “model wrapper,” or “model graft” of a given DE model file includes (1) DE model data or digital artifacts extracted or derived from the DE model file containing model metadata, and (2) locators or copies thereof for splice functions (e.g., API function scripts) applicable to the DE model data. Splice functions provide unified and standardized input and output API endpoints for accessing and manipulating the DE model data. The DE model data is model type specific, and the model splice is associated with model type specific input and output schemas. One or more different model splices may be generated from the same input DE model file(s), depending on the specific user application under consideration and the data access restrictions. Depending on the context, the shorter terms “splice,” “wrapper,” and / or “graft” may be used to refer to a spliced, wrapped, and / or grafted DE model. ● Model Representation: Within this disclosure, “Model Representation” of a given DE model includes any embodiment of the engineering model in the form of a collection of DE model files, model splices, and / or digital artifacts derived from the DE model. In some embodiments, the DE model representation includes a model type-specific locator to the DE model data and metadata, and potentially includes standardized input and output API endpoints for accessing and manipulating the DE model data. The descriptions relating to the use of model splices in this disclosure are also applicable to any other form of model representation. ● Model splicing or DE model splicing: The process for generating model splices from DE model files. DE model splicing encompasses human-readable document model splicing, where the DE model being spliced ​​is a human-readable text-based document. ● Model Splicer: Program code or script (uncompiled) that performs model splicing of a DE model. When applied to a specific DE model file of a given DE model type, a DE model splicer retrieves, extracts, or derives the DE model data associated with the DE model file, generates and / or encapsulates a splice function, and instantiates an API endpoint according to the input / output schema. ● Model splice linking: Generally refers to jointly accessing two or more DE model splices via an API endpoint or splice function. For example, data may be retrieved from one splice to update another (e.g., the input splice function of the first model splice calls the output splice function of the second model splice). Data may be retrieved from both splices to generate a new output (e.g., the output splice functions from both model splices are called). Data from a third splice may be used to update both the first and second splices (e.g., the input splice functions from both model splices are called). In this disclosure, "model linking" and "model splice linking" can be used interchangeably because linked model splices are mapped to correspondingly linked DE models. ● Digital Thread, Software-Defined Digital Thread, Software Code-Defined Digital Thread, Software Digital Thread, or Code Digital Thread: According to DAU, a digital thread is "a large, configurable, enterprise-level analytical framework for components that informs decision-makers throughout the system lifecycle by seamlessly facilitating the controlled interaction of trusted technical data, software, information, and knowledge in an enterprise data information knowledge system based on a digital system model template, providing the ability to access, integrate, and transform heterogeneous data into actionable information." Within the IDEP disclosed herein, a digital thread is a platform script that calls platform APIs to facilitate, manage, or orchestrate workflows through linked model splices to provide the aforementioned capabilities. That is, a digital thread within an IDEP is a script that connects data from one or more DE models, data sources, or physical artifacts to accomplish a particular mission or business objective, and may be referred to as a “software-defined digital thread” or “software digital thread,” implementing a communication framework or data-driven architecture that connects traditionally siloed DE models to enable a seamless flow of information between DE models via model splices. ● Tool linking: Similar to model splice linking, tool linking generally refers to jointly accessing two or more DE tools via a model splice, where a model splice function that encapsulates heterogeneous DE tool functionality is jointly called to execute DE tasks. ● Zero Trust ("ZT") Security: An information security principle that assumes no implicit trust between any element, agent, or user. Zero Trust can typically be implemented by implementing systematic mutual authentication and least privilege access through strict access control, algorithmic fairness, and data isolation. Within the IDEP disclosed herein, least privilege access through strict access control and data isolation can be implemented through model splicing and the IDEP system architecture. ● Hyperscale capability: The ability of a system architecture to scale appropriately when faced with massive demand. ● IDEP Enclave or DE Platform Enclave: A central command hub responsible for managing and functioning DE Platform operations. An enclave is an independent set of cloud resources that is partitioned to be accessed by a single customer (i.e., single-tenant) or market (i.e., multi-tenant) without depending on the resources of other enclaves. ● IDEP Exclave or DE Platform Exclave: A secondary hub located within the customer environment to support the customer's DE tasks and operations. An exclave is a set of cloud resources outside the enclave, managed by IDEP, for performing work for individual customers. An example of an exclave is a virtual machine (VM) and / or server maintained by IDEP to run DE tools for customers who may require such services. ● Digital Twin: According to DAU, a digital twin is "a virtual replica of a physical entity synchronized across time. Digital twins exist to replicate the configuration, performance, or history of a system. Two main subcategories of digital twins are digital instances and digital prototypes." A digital instance is "a virtual replica of the physical configuration of an existing entity. Digital instances typically exist to replicate the individual configurations of a product during construction or maintenance." A digital prototype is "an integrated multi-physical, multi-scale probabilistic model of a system design. Digital prototypes can use sensor information and input data to simulate the performance of the corresponding physical twin. Digital prototypes can exist before the physical counterpart materializes." Therefore, a digital twin is a real-time virtual replica of a physical object or system with a bidirectional flow of information between the virtual and physical domains. ● Reliable Twin: A reference design configuration at a given stage in the product lifecycle. In the design phase, the reliable twin is the twin configuration that best represents the design objectives. In the operational phase, the reliable twin is the twin configuration that best responds to actual field conditions, or "ground truth." ● Administrator or Administrator: A project manager or other authorized user. Administrators may have high-level privileges to create templates in the documentation system and manage settings in IDEP. ● Requester: A user who uses the platform to implement modeling and simulation for authentication and other purposes, and who can generate documentation in the digital documentation system, but who does not have administrator privileges to change the necessary templates, document formats, or other system settings. ● Reviewer / Approver: A user who reviews and / or approves templates, documents, or other system data. ● Contributors: Users who contribute to IDEP by providing comments or in other ways.

[0054] Interconnected Digital Engineering Platform (IDEP) Architecture Figure 1 shows an exemplary interconnected digital engineering platform (IDEP) architecture according to several embodiments of the present invention. IDEP 100 streamlines the product development process from concept to production by using a virtual representation or digital twin (DTw) 122 of the product to optimize and refine features before building a physical prototype or physical twin (PTw) 132, and by iteratively updating DTw 122 until DTw 122 and PTw 132 are synchronized to achieve desired performance targets for the product.

[0055] Specifically, manufacturers of products (e.g., airplanes, spacecraft, exploration rovers, missile systems, automobiles, railway systems, ships, remotely operated underwater vehicles, robots, drones, medical devices, biomedical devices, pharmaceutical compounds, drugs, power generation systems, smart grid measurement and management systems, microprocessors, integrated circuits, buildings, bridges, tunnels, chemical plants, oil and gas pipelines, refineries, etc.) may use the IDEP platform 100 to develop new products. Engineering teams from manufacturers may create or instantiate a digital twin (DTw) 122 of the product within a virtual environment 120 that includes detailed computer-aided design (CAD) models and finite element analysis (FEA) or computational fluid dynamics (CFD) simulations of component systems such as fuselages, wings, engines, propellers, tails, and aerodynamics. The DTw 122 virtually represents the design and performance characteristics of the product, allowing the team to optimize and refine its features before building a physical prototype 132 in the physical environment 130. In some embodiments, PTw132 may be an existing entity, while DTw122 is a digital instance that replicates the individual configurations of PTw132 in a built-out or maintained state. For illustrative purposes only, DTw122 and PTw132 are described in the context of building a new product, but those skilled in the art will understand that the instantiation of DTw122 and PTw132 may be performed in any order based on the specific use case under consideration.

[0056] The digital models (e.g., CAD models, FEA models, CFD models) used to create DTw122 are shown within the model plane 180 in Figure 1. The model plane 180 also shows a neural network (NN) model 184, which can provide machine learning-based predictive modeling and simulation for the DE process. DE models such as 182 can be spliced ​​into one or more model splices, such as 172 and 173, within the splice plane 170. Individual DTws, such as 122, are instantiated from the splice plane 170 via the application plane 160. Model splices, such as 172, can be linked to other model splices, such as 171, by platform scripts or application 162 on the application plane 160, forming digital threads. Multiple digital threads, such as 162 and 163, can be further linked across different stages or phases of the product lifecycle, from concept, design, test, to production. Digital threads further enable seamless data exchange and collaboration between departments and stakeholders, ensuring optimized, validated designs.

[0057] Model splicing provides input and output splice functions that can access and modify DE model data. Therefore, DE tasks associated with design updates and digital threads can be represented by scripted, interconnected, and pipelining tasks arranged in a directed acyclic graph (DAG) such as 124. An example of a DAG for DE tasks is illustrated in more detail with reference to Figure 10.

[0058] To enhance the design, external sensor data 140 can be collected, processed, and integrated into the application plane 160. This process involves linking data from different sources, such as physical sensors 134 on the prototype 132, physical environment sensors 136, and other external data streams, such as simulation data from the model plane 180. API endpoints provide access to digital artifacts from various environments (e.g., data from the physical twin (PTw) sensor 134) and integrate them into the splice plane 170 of the DTw 122. Model splices on the splice plane 170 enable autonomous data linkage and digital thread generation, ensuring that the DTw 122 accurately represents the real-world performance and characteristics of the product.

[0059] To verify the accuracy of DTw122, the engineering team may build or instantiate PTw132 based on the same twin configuration (i.e., digital design). The physical prototype 132 may be equipped with numerous sensors 134, such as accelerometers and temperature sensors, to collect real-time performance data. This data can be compared with simulations of DTw to verify the product's performance and validate its design.

[0060] Processed sensor data 144 can be used to estimate parameters that are difficult to measure directly, such as aerodynamic forces or tire contact force. Such processed sensor data provides additional data to DTw 122, further improving its accuracy and reliability. Processed sensor data 144 may be generated from physical environment sensors 136 using the physical environment 130, or it may be obtained from other external databases 142, as described below.

[0061] During development, feedback from customers and market research may be collected to identify potential improvements or adjustments to the product design. In the Analysis and Control Plane (ACP) 150, subject matter experts (SMEs) can analyze processed sensor data 144 and external expert feedback 114 to make informed decisions regarding necessary design changes. Such analysis 154 may be enhanced or made entirely possible by algorithms (i.e., static program code) or artificial intelligence (AI) modules. Linking of digital threads such as 162, physical sensors 134 and 136, processed sensor data 144, and expert feedback data 114 is performed in the ACP 150, and sensor and performance data are compared and analyzed to lead to modifications of the underlying model file through the digital threads.

[0062] In particular, sensor data 144 from the physical environment 130 and performance data 126 from the virtual environment 120 may be supplied to the comparison engine 152. The comparison engine 152 may include tools that enable platform users to compare various design iterations with each other and with design requirements, identify performance degradations and trends, and run verification and validation (V&V) tools.

[0063] Model splicing will be explained in more detail with reference to Figures 7-9 and 11-33. Model splicing enables the scripting of any DE operation, including the DE model file of model plane 180, with each DE model associated with heterogeneous, siloed DE tools. By coding DE models and DE operations using a unified corpus of scripts, IDEP 100 can become an aggregator where a large space of DE activities associated with a given product (e.g., airplanes, spacecraft, exploration rovers, missile systems, automobiles, railway systems, ships, remotely operated underwater vehicles, robots, drones, medical devices, biomedical devices, pharmaceutical compounds, drugs, power generation systems, smart grid measurement and management systems, microprocessors, integrated circuits, buildings, bridges, tunnels, chemical plants, oil and gas pipelines, refineries, etc.) can be threaded through program code. Thus, model splicing enables the linking and manipulation of all model files (e.g., 182, 184) associated with a given product within the same interconnected DE platform or DE ecosystem 100. As a result, the creation and training of AI modules for the purpose of manipulating DE models (e.g., 182), digital threads (e.g., 162), and digital twins (e.g., 122) becomes possible on a programmable and unified IDEP100.

[0064] Virtual and physical feedback loops Figure 1 illustrates the various stages of the product lifecycle using letter labels "A" through "H". At each stage, IDEP100 enables a feedback loop in which data originating from PTw or DTw is analyzed in ACP150, leading to the generation of a new twin configuration based on design modifications. The new twin configuration is stored in the twin configuration set and may be applied through the application and splice plane, thereby obtaining a modified model file registered in the digital thread.

[0065] The virtual feedback loop 104 begins with a decision 106 to instantiate a new DTw 122. The DAG of hierarchical task 124 enables the automatic instantiation of DTw 122 in the virtual environment 120 based on the twin configuration applied in process step 108 from the twin configuration set 156. DTw 122 and / or its components are then tested in the virtual environment 120, generating DTw performance data 126. Simultaneously, DTw 122 and / or its components may be tested and simulated on the model plane 180 using DE software tools, generating test and simulation performance data 174. The performance data 126 and 174 are combined, compared by the engine 152, and analyzed by the ACP 150, which may lead to the generation and storage of a new twin configuration. The virtual feedback loop 104 is completed by the final decision to instantiate DTw from the new twin configuration.

[0066] The physical feedback loop 102 begins with a decision 106 to instantiate a new PTw 132. The PTw 132 can be instantiated within the physical environment 130 from the model file of the model plane 180, associated with the applied twin configuration from the twin configuration set 156. The PTw 132 and / or its components are then tested within the physical environment 132, generating sensor data from the PTw sensor 134 and environmental sensor 136 located within the physical environment 130. This sensor data can be combined with data from an external database to obtain processed sensor data 144.

[0067] Data from the PTw sensor 134 may be directly added to the model file in the model plane 180 by the DE software tool used in the design process of the PTw 132. Alternatively, the PTw sensor data may be directly added to the digital thread 162 associated with the PTw 132 via the application plane 160. Furthermore, the processed sensor data 144 may be directly integrated into the IDEP 100 via the application plane 160. For example, the processed sensor data 144 may be sent to the ACP 150 for analysis, which may lead to the generation and storage of a new twin configuration. The physical feedback loop 102 is completed by the final decision to instantiate the PTw from the new twin configuration.

[0068] At each stage A through H of the product lifecycle, the system may be labeled as a single twin configuration as the current design criterion, and is described herein as the “reliable twin” or “reliable criterion.” The reliable twin represents the design configuration that best responds to the actual situation (i.e., ground truth). U.S. Provisional Patent Application No. 63 / 470,870 (IST-03.001P) provides a more complete description of reliable twins and their determination, which is incorporated herein by reference in whole.

[0069] With a faster feedback loop from sensor data and expert recommendations, the system updates DTw122 to reflect the latest design changes. This update process may include the engineering team analyzing feedback154 and implementing changes through IDEP100, or automated changes enabled by IDEP100, where updates to DTw122 are generated through programmed algorithms or AI modules. This iterative update process continues until DTw122 and PTw132 are synchronized and the product performance meets the desired targets. While IDEP100 itself may not specify a trustworthy criterion between DTw or PTw, the platform provides configurable mechanisms such as policies, algorithms, voting schemas, and statistical support, thereby allowing agents to designate a new DTw as a trusted DTw, or equivalently, specify when a PTw is a trustworthy and true source of information.

[0070] If significant design improvements are made, a new PTw prototype may be constructed based on the updated DTw. This new prototype will undergo further testing and validation to ensure that the product's performance and design are consistent with the project objectives.

[0071] Once DTw122 and PTw132 are validated and optimized, the product is ready for production. The digital thread connecting all stages of development can be queried via the splice plane 170 to generate documentation as needed to meet validation and verification requirements. The use of model splicing, along with the feedback architecture shown in Figure 1, improves the efficiency of the entire product innovation process.

[0072] Interconnected DE Platform and Product Lifecycle In Figure 1, the letter labels "A" through "H" indicate the following key steps in the product lifecycle according to several embodiments of the present invention. A. Digital models reside within the customer environment: A product can be represented by model files accessible through software tools located within the customer environment. Model plane 180 encompasses all model files associated with the product (e.g., 182). B. Preparation steps for design in the digital realm: Splice plane 170 encompasses model splices (e.g., 172) generated from the DE model file through model splicing. As will be explained in detail with reference to Figures 7-9 and 11-33, model splicing enables the integration and sharing of DE model files within a single platform. C. Linking threads between model splices as needed: To implement the product, model splices are linked through scripts in the application plane 160. A digital twin (DTw) 122 encompassing the product features as designed may be generated from the application plane 160 to run in a virtual environment 120. The complete twin configuration of the generated DTw is stored in a twin configuration set 156 located in the analysis and control plane (ACP) 150. Features or parts of the DTw 122 may be simulated in the model plane 180 using performance data 174 accessed through the splice plane 170. In one embodiment, features or parts of the PTw 132 or DTw 122 configuration may be simulated outside the platform, and the performance data is received by the ACP 150 for processing, as is the performance data 126 received from the DTw 122. D. Determining the "Design State": Performance data 126 from DTw122, or simulation performance data 174 obtained through the model plane 180 and accessed through model splicing, may be collected and sent to ACP150 for analysis. Performance data from different iterations of DTw122 may be compared to the design requirements via engine 152. Analysis of the differences may lead to the generation of new twin configurations stored in twin configuration set 156. Each twin configuration in twin configuration set 156 may be applied in the application plane 160 and splice plane 170 via process step 108 to instantiate the corresponding DTw. Multiple DTws may be generated and tested sequentially or simultaneously against the design requirements via comparison engine 152 and analysis module 154. Verification and validation tools may operate on various iterations of DTw. E. Determining the “Manufacturing State”: Once DTw122 meets the design requirements, the corresponding PTw132 prototype can be instantiated from the spliced ​​model file (e.g., 172). Sensor data may be collected from PTw134 or from within the physical environment 136 and combined with other external data 142 (e.g., sensor data from other physical environments). The resulting processed sensor data 144 may be sent to the analysis and control plane 150 and compared with performance data 126 from DTw and simulation (e.g., 174), leading to further iterations of DTw122 and PTw132, including a twin configuration set 156. The processed sensor data 144 can also be mapped to digital threads (e.g., 164) and model splices (e.g., 172) that manage the PTw132 tested through the application plane 160. F. Determining the “Assembled State”: Once the manufacturing process for the various parts as DTw and PTw is complete, the next step is to determine the assembled configuration. This involves creating a digital representation of the assembly to ensure that the assembly meets the specified requirements. The digital assembly takes into account the dimensions and tolerances of the parts in the “manufactured state.” To verify the feasibility of the digital assembly, testing is performed using measurement data obtained from the physical assembly and its individual components. The measurement data from the physical component parts serves as a reliable standard for the digital assembly, ensuring consistency with the real-world configuration. The digital assembly is compared to the actual physical assembly requirements to validate the assembled configuration. Subsequently, the testing and configuration of the digital assembly serve as a reliable standard for instructions to guide the physical assembly process and ensure accurate replication. The components of IDEP100 described above can be used in the assembly process. In its reliable iteration, DTw122 ultimately captures the precise details of the physical assembly, enabling comprehensive analysis and control in subsequent stages of the process. G. Determining the “Operating State”: Multiple digital twins 122 may be generated as needed to assess the performance of the physical assembly or its individual component parts. These digital twins are created based on specific performance metrics and function as virtual replicas of the physical system. The digital twins 122 are continuously updated and improved in real time using operational data (e.g., 144) collected from monitoring the performance of the physical assembly or its components. This data may, but is not limited to, processed sensor data, performance indicators, and other relevant information. By incorporating this real-time operational data, the digital twins 122 maintain synchronization with the actual system and provide an accurate representation of its operational performance. Any changes or improvements observed via sensor data 144 during the assembly's real-world operation are reflected in the DE model within the digital twin and recorded in the twin configuration set 156. This ensures that the digital twins remain up-to-date and consistent with the current state of the physical system. H. Predictive Analytics / Future Performance: The design process can be iteratively continued in a virtual environment 120 through new DTw122 configurations while the product is operating. Multiple digital twins can be created to evaluate the future performance of a physical assembly or its components based on specific performance metrics. Simulations are performed with various control policies to assess their impact on performance targets and costs. The results of these simulations help determine which specific control policies should be implemented (e.g., tail volume coefficient and sideslip angle for an aircraft product). Digital twin DE models (e.g., 182) are continuously updated and refined using the latest sensor data, control policies, and performance metrics to improve predictive accuracy. This iterative process ensures that the digital twins (e.g., 122, 156) provide reliable predictions of future performance and support informed decision-making.

[0073] The hardware components that make up IDEP100 (e.g., servers, computing devices, storage devices, network links) can be centralized or distributed across various entities, including one or more DE service providers and DE clients, as further illustrated in the context of Figures 3 and 4. Figure 4 shows examples of various potential configurations for instantiating the DE platform within a customer's physical systems and information technology (IT) environment, which is typically a virtual private cloud (VPC) protected by a firewall.

[0074] DE documentation using live or magic documents The methods and systems described herein enable the updating and generation of DE documents using all the functions of IDEP shown in Figure 1. In Figure 1, the IDEP virtual feedback loop 104 enables the scripting of program code within the digital thread 162 for the generation, storage, and updating of the digital twin 122 and twin configuration 156. Similarly, the IDEP virtual feedback loop 104 also enables the scripting of program code within the digital thread 162 for the generation, storage, and updating of DE documents. This enables the creation and maintenance of so-called live digital engineering documents.

[0075] Live DE documents are more similar to DTws than traditional static documents in that they are configured to be continuously updated through a digital thread to reflect the latest changes within a particular twin configuration. In particular, trusted live DE documents are configured to reflect the latest trusted twin configuration. "Printing" a live DE document corresponds to generating a frozen (i.e., static) timestamped version of the live DE document. Thus, "printing" a live DE document is equivalent to "instantiation" in DTw.

[0076] Live DE documents can also be known as magic documents because changes made within the twin configuration (for example, through modifications to model files) may instantly appear in the relevant data fields and sections of the live DE document. Similarly, reliable live DE documents can also be known as reliable magic documents because they continuously reflect data from the reliable twin and therefore always represent a reliable and true source.

[0077] Given the large amount of data and potential modifications performed during the product lifecycle, scripts implementing live DE documentation may be configured to account for a predefined maximum delay between modifications to model files and the execution of corresponding changes in the live DE documentation. Furthermore, for similar reasons, scripts implementing live DE documentation are limited to operations on a specified subset of model files within the DTw, and therefore reflect only changes to critical parameters and configurations of the DTw.

[0078] In one embodiment of the present invention, an IDEP script (e.g., an IDEP application) that accesses model data via one or more model splices and DE document templates for creating and / or updating a live DE document may dynamically update the live DE document using a software-defined digital thread on the IDEP platform. In such an embodiment, the IDEP script may dynamically receive user interactions. In response to a user updating data for the model and / or specific parameter settings, the IDEP script may dynamically propagate the user's updates to the DE document through the corresponding digital thread.

[0079] In another embodiment of the present invention, an IDEP script may instantiate a DE document having sufficient specifications to generate a physical twin (PTw). In such an embodiment, the IDEP script may receive a digital twin configuration of a physical twin, generate a live DE document associated with the digital twin configuration, receive a predetermined timestamp, and generate a printed DE document (i.e., a statically timestamped version of the live DE document at the predetermined timestamp). Such operation is sometimes referred to as "printing the digital twin."

[0080] In yet another embodiment of the present invention, when an IDEP script detects an update, it may instantiate (i.e., "print") a DE document specifying the updated digital twin. In such an embodiment, the IDEP script may detect modifications to the DE model or associated digital thread. In response to detecting a modification, the IDEP script may update the relevant data fields and sections of the live DE document based on the detected modification, and always generate an updated printed DE document having the updated relevant data fields and sections based on the updated live DE document.

[0081] In various embodiments, a software code definition digital thread may be associated with a companion magic document (or "magic doc") that provides explainability and enables an audit trail for the digital thread. This "magic document" may be generated with the help of AI and describes the process by which the digital thread efficiently translates user intent into an orchestration script that includes relevant model splices and splice functions. Specifically, a magic document generated by IDEP may describe an embodiment of the digital thread of user intent and may include pseudocode, scripts, data fields, and natural language-based descriptions. When the digital thread and its associated orchestration script are executed to perform a DE task, the magic document may record the completion of the task for auditability. The digital thread may sequentially include the orchestration script. One or more corresponding magic documents for a digital thread may call a subset of data points and orchestration script examples as needed. In some embodiments, a script-generating ML model that receives input pseudocode or detailed user instructions derived from user intent can be trained with previous IDEP digital threads, documents, and optionally, IDEP platform API documentation. In addition to generating digital threads (with orchestration scripts and comments), the script-generating ML model can also be configured to generate a magic dog that explains how the generated digital threads address user intent.

[0082] In some embodiments, receiving user interactions with a DE model, modifications to a DE model, or modifications to an associated digital thread may be performed through a push configuration, in which case the model splicer or digital thread script sends any relevant updates occurring to the IDEP script immediately or within a specified maximum delay time. In other embodiments, receiving user interactions with a DE model, modifications to a DE model, or modifications to an associated digital thread may be performed through a pull configuration, in which case the model splicer or digital thread script flags recent modifications until the IDEP script queries the associated DE model or associated digital thread (via the model splice) for flagged modifications. In these embodiments, the IDEP script may extract modified information from the modified DE model or modified digital thread (via the model splice) to update the live DE document. In further embodiments, receiving user interaction with a DE model, a modification of a DE model, or a modification of an associated digital thread may be performed through a pull configuration, and the IDEP script periodically checks the relevant DE model or associated digital thread (via model splicing) against the modified data fields by comparing the data found in the live DE document with the periodically extracted model and digital thread data. In these embodiments, the IDEP script may update the live DE document with the modified data.

[0083] Dynamic document updates Some embodiments described herein focus on documentation, or the preparation and updating of documents, as well as document management (e.g., for review). As described, some embodiments of the system enable dynamic updates to documents, relating to software-defined digital threads in the IDEP platform and associated documentation.

[0084] It has been proposed to use an ML engine along with model data and templates to create and / or update documents almost instantaneously as a single action. Furthermore, the digital engineering platform dynamically interacts with the user. When a user interacts with the system and updates model data or specific parameter settings, these changes can be propagated to the associated documentation through the corresponding digital thread. The AI ​​architectures involved include locally instantiated large-scale language models (LLMs, for data security reasons) and non-LLM approaches (e.g., NLP-based) to create, update, or predict documentation in the form of sentences, paragraphs, and entire documents. At the same time, attempting to update the entire system of digital threads with each update can be extremely slow and may pose a security risk to the system. Therefore, it may be more efficient to generate live DE documents that are updated within the maximum latency based on a subset of the system's DE models.

[0085] Interconnected Digital Engineering and Certification Ecosystem Figure 2 shows exemplary embodiments of IDEP as an interconnected digital engineering (DE) and certification ecosystem 200 according to several embodiments of the present invention, as well as exemplary digitally certified products. The interconnected DE and certification ecosystem 200 can be seen as a specific instance or implementation of IDEP 100 shown in Figure 1. IDEP may also be referred to as the “DE metaverse”.

[0086] The Interconnected DE and Certification Ecosystem 200 is a computer-based system that links models and simulation tools with their associated requirements to fulfill the purposes of verification, validation, and certification. Verification refers to a method of evaluating whether a product, service, or system meets specified requirements and is suitable for its intended purpose. For example, in the aviation industry, the verification process may include testing aircraft components to ensure that they can withstand the forces and conditions encountered during flight. Verification also includes externally checking customer or stakeholder needs. Validation refers to a method of evaluating whether the overall performance of a product, service, or system is suitable for its intended use, including compliance with regulatory requirements, and its ability to meet the needs of its intended users. Validation also includes internal checks against specifications and regulations. The Interconnected DE and Certification Ecosystem 200 disclosed herein is designed to connect and bridge a large number of heterogeneous DE tools and models from numerous engineering domains and fields, or from separate organizations that may wish to share models with each other but have no other interaction. In various embodiments, the system implements a robust, scalable, and efficient DE model collaboration platform using an extensible model splice with widely distributed DE model types and DE tool data structures and associated functionalities, an application layer for linking or connecting DE models via APIs, digital threads for connecting live engineering model files for collaboration and sharing, digital documentation management to assist in preparing engineering and certification documents suitable for verification and validation (V&V) purposes, and AI assistance with the functionalities of the aforementioned system components.

[0087] More specifically, Figure 2 shows an example of an interconnected DE and certification ecosystem, as well as examples of digitally certified products 212A, 212B, and 212C (collectively referred to as digitally certified products 212). For example, in some embodiments, digitally certified product 212A may be an unmanned aerial vehicle (UAV) or other aircraft, digitally certified product 212B may be a drug or other chemical or biological compound, and digitally certified product 212C may be a process, such as a manufacturing process. Generally, digitally certified products 212 may include any product, process, or solution that can be developed, tested, or certified (partially or entirely) using DE tools such as 202. In some embodiments, digitally certified products 212 may not be limited to physical products, but may include non-physical products such as methodologies, processes, and software. Physical systems and systems that physically interact often require multiple DE tools to assess compliance with common V&V products, simply for modeling and simulation (M&S) needs. However, many complex non-physical systems may also require multiple DE tools for product development, testing, and / or certification. With this in mind, various other possibilities for digitally certified products will be recognized by those skilled in the art. By including regulatory and certification standards, compliance, calculations, and tests (for example, for product and / or solution development, testing, and certification), users can directly incorporate relevant regulatory and certification standards, compliance, calculations, and test data into their DE workflows. Regulatory and certification standards, compliance, calculations, and tests are sometimes referred to herein as “common validation and verification (V&V) products.”

[0088] The digital authentication product 212 in Figure 2 may be designed and / or authenticated using an interconnected DE and authentication ecosystem 200. The interconnected DE and authentication ecosystem 200 may include a user device 206A, an API 206B, or other similar human-to-machine or machine-to-machine communication interface operated by the user. The user may be a human 204 with varying skill levels, or an artificial user such as an algorithm, artificial intelligence, or other software that interfaces with the ecosystem 200 through API 206B. The ecosystem 200 may further include a computing and control system 208 (hereinafter, "computing system 208") connected to and / or including a data storage unit 218, an artificial intelligence (AI) engine 220, and an application and service layer 222. In some embodiments, the artificial intelligence (AI) engine 220 is a machine learning (ML) engine. References to "machine learning engine 220" or "ML engine 220" may be extended more generally to artificial intelligence (AI) engine 220. For clarity, any user selected from a variety of potential human or artificial users will be simply referred to herein as User 204. In some embodiments, the computing system 208 may be a centralized computing system. In some embodiments, the computing system 208 may be a distributed computing system. In some cases, User 204 may be considered part of the ecosystem 200, while in other embodiments, User 204 may be considered separate from the ecosystem 200. The ecosystem 200 may include one or more DE tools 202, such as data analysis tools 202A, computer-aided design (CAD) and finite element analysis (FEA) tools 202B, simulation tools 202C, drug modeling and simulation (M&S) tools 202D-202E, and manufacturing M&S tools 202F-202G.Ecosystem 200 may also include a repository of common V&V products 210, such as regulatory standards 210A-210F related to UAV development and certification, medical standards 210G (e.g., CE marking (Europe), FCC Declaration of Conformity (USA), IECEE CB scheme (Europe, North America, parts of Asia and Australia), CDSCO (India), FDA (USA), etc.), medical certification rules 210H (e.g., ISO13485, ISO14971, ISO9001, ISO62304, ISO10993, ISO15223, ISO11135, ISO11137, ISO11607, IEC60601, etc.), manufacturing standards 210I (e.g., ISO9001, ISO9013, ISO10204, EN1090, ISO14004, etc.), and manufacturing certification rules 210J (e.g., General Conformity Certification (GCC), etc.).

[0089] In Figure 2, the computing system 208 is centrally located within the architecture and configured to communicate with (e.g., receive data from and send data to) user devices 206A, or APIs 206B such as APIs associated with artificial users, DE tools 202 via APIs or software development kits (SDKs) 214, and a repository 210 of common V&V products via API / SDK interfaces 216. For example, the computing system 208 may communicate with user devices 206A and / or API 206B to send or receive data corresponding to design prototypes, user information (e.g., user credentials), engineering-related inputs / outputs associated with DE tools 202, digitized common V&V products, product design evaluations, user instructions (e.g., search requests, data processing instructions, etc.). The computing system 208 may also communicate with one or more DE tools 202 to send engineering-related inputs for performing analysis, modeling, simulation, testing, etc., and receive engineering-related outputs associated with the results. The computing system 208 may also be configured to communicate with the Common V&V Product Repository 210 to retrieve data corresponding to one or more digitized Common V&V Products 210 and / or upload new Common V&V Products to the Common V&V Product Repository 210, such as those received from user 204. All communications may be transmitted and verified securely, for example, using methods that rely on zero-trust security. In some embodiments, the ecosystem's computing system may interface with regulatory and / or certification authorities (for example, via a website operated by the authority) to retrieve digitized Common V&V Products published by regulatory authorities that may be relevant to the product the user is designing. In some embodiments, the user may upload digitized Common V&V Products to the ecosystem itself.

[0090] The computing and control system 208 may process and / or store incoming data to perform analytical and control functions, and in some embodiments, as further described herein, may access a machine learning engine 220 and / or an application and service layer 222 to identify useful insights based on the data. Centering the computing system 208 within the ecosystem architecture offers many advantages, including reducing the technical complexity of integrating various DE tools, improving the user's product development experience, intelligently connecting common V&V products such as standards 210A-210F to the DE tools 202 most useful for meeting the requirements associated with the common V&V products, and enabling monitoring, storage, and analysis of various data flowing between elements of the ecosystem throughout the entire product development process. In some embodiments, data flowing through and potentially stored by the computing system 208 may also be auditable for purposes such as preventing security breaches and performing data quality controls. Similarly, any analytical and control functions performed via the computing system 208 may be trackable for auditability and traceability considerations.

[0091] Referring to one specific example shown in Figure 2, user 204 may use the DE and certification ecosystem to produce a digitally certified UAV 212B. For example, user 204 may be primarily interested in certifying the UAV as meeting the requirements of a specific regulatory standard 210E related to the failure conditions of the UAV (e.g., "MIL-HDBK 516C 4.1.4 - Failure Conditions"). In this use scenario, user 204 may develop a digital prototype of the UAV on user device 206A or using API 206B and send the prototype data to computing system 208 (e.g., as at least one of a CAD file, MBSE file, etc.). Along with the prototype data, user 204 may transmit additional data via user device 206A, including indications of common V&V products that user 204 is interested in authenticating (e.g., regulatory standard 210E), user credentials information for accessing one or more capabilities of computing system 208, and / or instructions for running one or more digital models, tests, and / or simulations using a subset of DE tools 202.

[0092] Referring to another example shown in Figure 2, user 204 may use the DE and certification ecosystem to produce a digitally certified drug, compound, or biologic 212A. For example, user 204 may be primarily interested in certifying the drug, compound, or biologic 212A as meeting the requirements of specific medical standards 210G and medical certification rules 210H. In this use scenario, user 204 can develop a digital prototype of the drug, compound, or biologic on user device 206A or using API 206B, and send the prototype data (e.g., as a molecular modeling file) to computing system 208. Along with the prototype data, user 204 may transmit additional data via user device 206A, including indications of common V&V products that user 204 is interested in authenticating (e.g., medical standards 210G and medical certification rules 210H), user credentials information for accessing one or more capabilities of computing system 208, and / or instructions for running one or more digital models, tests, and / or simulations using a subset of DE tools 202 (e.g., drug M&S tools 202D-202E).

[0093] Referring to yet another example shown in Figure 2, user 204 can use the digital engineering and certification ecosystem to create a digitally certified manufacturing process 212C. For example, user 204 may be primarily interested in certifying the manufacturing process 212C as meeting the requirements of a specific manufacturing standard 210I and manufacturing certification rule 210J. In this use case, user 204 can develop a digital prototype of the manufacturing process on user device 206A or using API 206B and send the prototype data to computing system 208. Along with the prototype data, user 204 may transmit additional data via user device 206A, including indications of common V&V products that user 204 is interested in authenticating the process (e.g., manufacturing standard 210I and manufacturing certification rule 210J), user credentials information for accessing one or more capabilities of computing system 208, and / or instructions for running one or more digital models, tests, and / or simulations using a subset of DE tools 202 (e.g., manufacturing M&S tools 202F-202G).

[0094] In any of the examples described above, the computing system 208 can receive data transmitted from the user device 206A and / or API 206B, process the data, and evaluate whether the common V&V product in question (e.g., regulatory standard 210E, medical standard 210G, medical certification rule 210H, manufacturing standard 210I, manufacturing certification rule 210J, etc.) is met by the user's digital prototype in the context of the analysis and control plane 150 shown in Figure 1. For example, this may involve communicating with the common V&V product repository 210 via API / SDK 216 to retrieve the relevant common V&V product in question, and processing the regulatory and / or certification data associated with the common V&V product to identify one or more requirements for a UAV prototype, a drug, compound, or biologic prototype, a manufacturing process prototype, etc. In some embodiments, the repository 210 for common V&V products may be hosted by a regulatory and / or certification authority (or another third party), and retrieving regulatory and / or certification data may involve interfacing with one or more data resources maintained by the regulatory and / or certification authority (or another third party) using the API / SDK 216. In some embodiments, regulatory and / or certification data may be provided directly by user 204 (e.g., along with prototype data) via user device 206A and / or API 206B.

[0095] Evaluating whether the target common V&V product is met by the user's digital prototype may also involve processing the prototype data received from the user device 206A or API 206B to determine whether one or more identified requirements are actually met. In some embodiments, the computing system 208 may include one or more plug-ins, local applications, etc., for processing the prototype data directly on the computing system 208. For example, model splicing and digital threading applications will be described in detail later with reference to Figures 6-9. In some embodiments, the computing system may simply preprocess the received prototype data (for example, to derive input to the DE tool 202) and then send instructions and / or input data to a subset of the DE tool 202 via the API / SDK 214 for further processing.

[0096] Not all DE tools 202 are necessarily required to meet specific regulations and / or certification standards. Therefore, in the example of the UAV provided in Figure 2, the computing system 208 may determine that only the data analysis tool 202A and the finite element analysis tool 202B are required to meet the regulatory standard 210E for failure conditions. In the example of a drug, compound, or biologic provided in Figure 2, the computing system 208 may determine that only the drug M&S tools 202D-202E are required to meet the medical standard 210G and the medical certification rule 210H. In the example of a manufacturing process provided in Figure 2, the computing system 208 may determine that only the manufacturing M&S tools 202F-202G are required to meet the manufacturing standard 210I and the manufacturing certification rule 210J. In other embodiments, if user 204 is a qualified subject matter expert (SME), user 204 may identify the specific subset of DE tools 202 that should be used to meet the common V&V product in question. In other embodiments, user 204 may input several proposed DE tools 202 to satisfy the common V&V product of interest into computing system 208, and computing system 208 may recommend a modified subset of DE tools 202 to user 204 for final approval, provided that user 204 is a qualified SME. After the subset of DE tools 202 has been identified, computing system 208 may then send instructions and / or input data to the identified subset of DE tools 202 to perform one or more models, tests, and / or simulations. The results of these models, tests, and / or simulations (or “Engineering-Related Data Outputs” or “Digital Artifacts”) may be returned to and received by computing system 208.

[0097] In yet another embodiment, user 204 may input the necessary DE tools, such as 202F, to satisfy common V&V product 210I, and computing system 208 may determine that another DE tool, such as 102G, is also required to satisfy common V&V product 210I. The computing system may then send instructions and / or input data to both DE tools (e.g., 202F and 202G), and the outputs of these DE tools may be transmitted and received by computing system 208. In some cases, the input data submitted to one of the DE tools (e.g., 202G) may be derived (e.g., by computing system 208) from the output of the other DE tool (e.g., 202F).

[0098] After receiving engineering-related data output or digital artifacts from the DE tool 202, the computing system 208 may then process the received engineering-related data output to evaluate whether the requirements identified for the common V&V product in question (e.g., regulatory standard 210E, medical standard 2110G, medical certification rule 210H, manufacturing standard 210I, manufacturing certification rule 210J, etc.) are met. For example, the application and service 222 may provide instructions for orchestrating validation or verification activities. In some embodiments, the computing system 208 may generate a report summarizing the evaluation results and send the report to device 206A or API 206B for review by user 204. If all requirements are met, the prototype can be certified, resulting in a digitally certified product 212 (e.g., a digitally certified drug, compound, or biologic 212A, a digitally certified UAV 212B, a digitally certified manufacturing process 212C, etc.). However, if some regulatory requirements are not met, additional steps may need to be taken by User 204 to certify the product prototype. In some embodiments, the report sent to the user may include recommendations for these additional steps (e.g., suggestions for one or more design changes, suggestions for replacing one or more components with previously designed solutions, suggestions for one or more adjustments to the inputs for models, tests, and / or simulations). If the requirements for the common V&V product are partially met or exceed the collective capabilities of the distributed engineering tools 202, the computing system 208 may provide User 204 with a report recommending partial certification, compliance, or performance of the common V&V product (e.g., digital certification of a subsystem or subprocess of the prototype). The process for generating recommendations for User 204 is described in more detail below.

[0099] In response to the report review, user 204 can make local design changes to the digital prototype and / or send one or more instructions to computing system 208 via user device 206A or API 206B. These instructions may include, for example, instructions for computing system 208 to re-evaluate the updated prototype design, use one or more different DE tools 202 for the evaluation process, and / or modify the inputs to DE tools 202. Computing system 208 then receives the user instructions, performs one or more additional data operations in accordance with these instructions, and may provide user 204 with an updated report. Through this iterative process, user 204 can leverage the interconnected digital engineering and certification ecosystem to design and ultimately certify (for example, by providing certification compliance information) prototypes (e.g., UAV prototypes, drug prototypes, manufacturing process prototypes, etc.) with respect to the common V&V product of interest. Importantly, because all of these steps occur in the digital world (e.g., using digital prototypes, digital models / tests / simulations, and digital certifications), a considerable amount of time, cost, and material can be saved compared to processes involving physical prototyping, evaluation, and / or certification, such as similar UAVs, drugs, and manufacturing processes. If the requirements associated with the common V&V product are partially met or exceed the collective capabilities of DE tool 202, computing system 208 may provide user 204 with a report recommending partial certification, compliance, or performance (e.g., digital certification of a subsystem or subprocess of a prototype) of a subset of the common V&V product.

[0100] While the above example focuses on the use of an interconnected digital engineering and authentication ecosystem by a single user, additional benefits of the ecosystem can be realized through repeated use by multiple users. As mentioned above, by placing computing system 208 at the center of the ecosystem architecture, computing system 208 can monitor and store various data flows through the ecosystem. Therefore, as the number of users utilizing the ecosystem for digital product development increases, data associated with each use of the ecosystem can be stored (e.g., in storage 218), tracked (e.g., with metadata), and analyzed to yield various insights. These insights can be used to further automate the digital product development process and make it easier to navigate the digital product development process for non-subject experts.

[0101] In fact, in some embodiments, user credentials for user 204 can indicate user 204's skill level and control the amount of automated assistance provided to the user. For example, non-subject experts may only be permitted to use the ecosystem to browse pre-made designs and / or solutions, use DE tool 202 with specific default parameters, and / or follow a predetermined workflow while automated assistance guides user 204 through the product development process. On the other hand, more skilled users may still be provided with automated assistance, but may be given more opportunities to override default or suggested workflows and settings.

[0102] In some embodiments, the computing system 208 may host applications and services 222 that automate or partially automate expected or common data transmissions from users 204, including components of common V&V products and data transmission components; expected or common interfaces and / or data exchanges between various DE tools 202, including interface components; expected or common interfaces and / or data exchanges with machine learning (ML) models implemented on the computing system 208 (e.g., models trained and / or implemented by the ML engine 220); and expected or common interfaces and / or data exchanges between applications and services themselves (e.g., within the application and service layer 222).

[0103] In some embodiments, data from multiple uses of the ecosystem (or a portion of the data described above) may be aggregated to develop a training dataset. For example, usage records 217 collected via computing system 208 may be despecificated or anonymized before being added to the training set. Such usage records may include model parameters and metadata, tool configurations, common V&V products that match a particular model or tool, user interactions with the system including inputs and actions, and other user-defined or system-defined configurations or decisions when using the ecosystem for digital engineering and authentication. For example, an exemplary despecificated usage record may include a combination of a specific DE tool, a specific target metric, a specific quantitative deviation, and a corresponding specific user update to the DE tool under this configuration. Another exemplary despecificated usage record may include a user-specific subset of DE tools 202 that should be used to meet the common V&V product in question.

[0104] This training dataset can then be used to train an ML model (for example, using the ML engine 220) to learn the steps and actions about the certification process, and to perform various tasks including identifying which DE tool 202 to use to satisfy specific common V&V products, identifying specific models, tests, and / or simulations (including their inputs) to be performed using DE tool 202, identifying common V&V products that need to be considered for specific types of products, identifying one or more recommended actions for user 204 to be taken in response to failure to meet regulatory requirements, and estimating the sensitivity of the model / test / simulation to specific inputs. The output of the trained ML model can be used to implement various features of the interconnected digital engineering and certification ecosystem, including automatically suggesting inputs (e.g., inputs to DE tool 202) based on previously performed inputs, predicting time and cost requirements for developing a product, predictively estimating the results of sensitivity analysis, and further suggesting design changes, original designs, or design alternatives to the user's prototype (e.g., via assistive AI or generative AI) to overcome one or more requirements (e.g., regulatory and / or certification requirements) associated with common V&V products. In some embodiments, with sufficient training data, the ML engine 220 may independently generate new designs, models, simulations, tests, common V&V products, and / or digital threads based on data collected from multiple uses of the ecosystem. Furthermore, such new designs, models, simulations, tests, common V&V products, and digital threads generated by the ML engine 220 may be added to the training set for further fine-tuning of the ML algorithm in a reinforcement learning setup, once approved and refined by the user.

[0105] As illustrated in the context of Figures 7-9 and 11-33, the aforementioned collection of training datasets, as well as the training of ML and AI modules, including the ML engine 220, can be enabled by model splicing techniques. The model splicing described herein enables the scripting of DE model behavior, encompassing heterogeneous DE tools within a corpus of prescriptive program code, and facilitates the code-definition digital threading of a large space of DE activities involving DE models across different domains. Using ML and AI techniques, scripts may be created to perform virtually any DE task and any digital thread, enabling programmable and machine-learnable dynamic changes to DE model files, digital threads, and ultimately digital or physical twins throughout the product lifecycle. For example, in the embodiment shown in Figure 2, the ML engine 220 can manage or orchestrate interactions between spliced ​​DE models, DE tools, and common V&V products (e.g., DE requirements) based on user intent and input-specific digital thread options. Sample DE tasks that can be performed by the ML engine 220 include, but are not limited to, (1) aligning models / analyses to authentication lifecycle requirement steps, (2) optimizing computations by determining appropriate fidelity for each model, (3) optimizing computational resources for a specific tool / model, or (4) optimizing computational resources across multiple models. The ML-enabled execution of DE tasks is not limited to authentication or resource optimization, but encompasses the entire DE operation space. Rather, the ML engine 220 can function as an AI multiplexer for the DE platform.

[0106] In addition to storing data used to enable the development of ML models, previous prototype designs and / or solutions (e.g., previously designed components, systems, models, simulations, and / or other engineering representations thereof) may be stored within the ecosystem (e.g., within storage 218) to enable users to search for and build upon the work of other users. For example, previously designed components, systems, models, simulations, and / or other engineering representations thereof may be searched by user 204 and / or suggested to user 204 by computing system 208 to meet one or more requirements associated with a common V&V product. Previously designed components, systems, models, simulations, and / or other engineering representations thereof may be used as is by user 204 or as a starting point for additional modifications. This store or repository of previously designed components, systems, models, simulations, and / or other engineering representations (whether they are ultimately certified or not) may be monetized to create a marketplace for digital products, which may be used to save time during the digital product development process, to allow users to recall alternative design ideas, and to avoid redundant efforts. In some embodiments, data corresponding to previous designs and / or solutions may be stored only if the user who developed the design and / or solution chooses to share the data. In some embodiments, the repository of previous designs and / or solutions may be containerized for private use within a single company, team, organizational entity, or technical field for private use (for example, to avoid the undesirable disclosure of confidential information). In some embodiments, user credentials associated with user 204 may be checked by computing system 208 to determine which designs and / or solutions stored in the repository are accessible to user 204.In some embodiments, the use of previously designed components, systems, models, simulations, and / or other engineering representations thereof may be available only to other users who pay a fee for their use.

[0107] Exemplary IDEP implementation architecture with services and features Figure 3 shows another exemplary embodiment of IDEP, illustrating the services and features provided by several embodiments of the present invention. Specifically, Figure 3 shows an exemplary implementation architecture, 300, which includes several exemplary components, namely, an IDEP enclave 302, a cloud service 304, and a customer environment 310, which optionally includes an IDEP exclave 316. This exemplary architecture 300 of IDEP is designed in accordance with zero-trust security principles and is further designed to support scalability and robust, resilient operation. Both the IDEP enclave 302 and the IDEP exclave 316 instantiate IDEP 100 shown in Figure 1, and the IDEP exclave 316 implements model splicing and splice plane 170 in several embodiments of the present invention. An enclave is an independent set of cloud resources that are partitioned to be accessed by a single customer (i.e., single-tenant) or market (i.e., multi-tenant) without depending on the resources of other enclaves. An exclave is a set of cloud resources outside of an enclave, managed by IDEP, for performing work for individual customers. An example of an exclave would include virtual machines (VMs) and / or servers that IDEP maintains to run DE tools for customers who require such services.

[0108] In particular, the IDEP enclave or DE platform enclave 302 may function as the origin of services rendered by IDEP and may be visualized as a central command and control hub responsible for managing and orchestrating all platform operations. For example, enclave 302 may be implemented using the interconnected DE and authentication ecosystem computing system 208 shown in Figure 2. The DE platform enclave 302 is designed to integrate both a zero-trust security model and hyperscale capabilities, resulting in a secure and scalable processing environment tailored to individual customer needs. Zero-trust security features include, but are not limited to, strict access control, algorithmic fairness, and data isolation. Enclave 302 also supports ML engines such as 220 for real-time analytics, auto-scaling features for workload adaptability, and API-based interoperability with third-party services. Security and resource optimization are enhanced through multi-tenant support, role-based access control, and data encryption both at rest and in transit. The DE platform enclave 302 may also include one or more of the features described below.

[0109] Firstly, the IDEP Enclave 302 may be designed in accordance with zero-trust security principles. In particular, the DE Platform Enclave 302 may adopt zero-trust principles to ensure that no implicit trust is assumed between any elements within the system, such as the digital model, platform agents, or individual users (e.g., user 204) or their actions. That is, there are no agents that can inherently be trusted, and the system can always authenticate or authorize for a particular job. The model is further reinforced through a strict access control mechanism, restricting even the management team (e.g., a team of individuals associated with the platform provider) to predetermined limited access to the enclave resources. To further enhance this robust security stance, data encryption is applied both at rest and in transit to effectively mitigate the risk of unauthorized access and data breaches.

[0110] IDEP Enclave 302 can also be designed to maintain isolation and independence. A key aspect of the Enclave's architecture is its focus on fairness and isolation. DE Enclave 302 enforces a strong isolation policy by not allowing cryptographic dependencies from external Enclaves. The Enclave's design also allows for both single-tenant and multi-tenant configurations, further enhancing data and process isolation between customers 306 (e.g., user 204). Additionally, DE Enclave 302 is designed with a decoupled set of resources, minimizing interdependencies and thereby promoting system efficiency and autonomy.

[0111] The IDEP Enclave 302 can also be designed to better accommodate changing operating requirements for scalability and adaptability. For example, the Enclave 302 can incorporate hyperscale-like characteristics in conjunction with zero-trust principles to enable scalable growth and effectively handle high-performance workloads.

[0112] IDEP Enclave 302 can also be designed to accommodate various customer workflows and DE models through a rigorous access control mechanism for workflow adaptability. This configurability enables a modular approach that integrates different functions from data ingestion to algorithm execution without compromising a zero-trust security posture. The adaptability of Platform 300 ensures high versatility for numerous use cases while maintaining consistent performance and robust security.

[0113] The IDEP Enclave 302 may be further designed to enable analytics for robust platform operation. The core of the Enclave's operational efficiency is a machine learning engine (e.g., machine learning engine 220) capable of performing real-time analytics. This improves decision-making and operational efficiency across the entire Platform 300. An auto-scaling mechanism may also be included to enable dynamic resource allocation based on workload demands, further enhancing the platform's responsiveness and efficiency.

[0114] In the exemplary embodiment shown in Figure 3, the IDEP enclave 302 includes several components, which will be described in further detail herein.

[0115] The "Monitoring Service Cell" may provide "Monitoring Services" and "Telemetry Services." A cell may refer to a set of microservices, for example, a set of microservices running within a Kubernetes pod. These components are focused on maintaining, tracking, and analyzing the performance of Platform 300 to ensure good service delivery, including advanced machine learning capabilities for real-time analysis. The "Search Service Cell" provides "Search Services" to help efficiently retrieve information from DE Platform 300 and add it to its overall functionality. The "Logging Service Cell" and "Control Plane Service Cell" provide "Logging Services," "File Services," and "Job Services" to record and manage operational events and information flows within Platform 300 that assist in the functionality of Platform 300. The "Static Asset Service Cell" provides "Static Services" and may house user interfaces, SDKs, command-line interfaces (CLI), and documentation for Platform 300. The "API Gateway Service Cell" may provide the "API Gateway Service" and may provide DE Platform APIs (or APIs 214, 216) (e.g., APIs 214, 216), and may act as an intermediary for requests between client applications (e.g., DE Tools 202, Common V&V Product Repository 210, etc.) and the Platform Service. In some embodiments, the API Gateway Service Cell may receive and respond to requests from agents such as the DE Platform Exclaver 316 in order to provide splice functions for model splicing purposes.

[0116] As shown in Figure 3, the architecture of the DE platform 300 may also include cloud services 304 that provide services that can modify software for orchestration of DE platform operation, although they cannot interact with customer data. In exemplary embodiments, several cloud resources provide support and basic services to the platform. For example, in the embodiment of the DE platform 300 shown in Figure 3, cloud services 304 include a "Customer Identity and Access Management (IAM) service" that ensures secure and controlled access to platform 300. Cloud services 304 also include a "Testing service" that tests tools for validating the operation of the platform. Cloud services 304 may also include an "Orchestration service" that controls and manages the lifecycle of containers on platform 300. Cloud services 304 may also include "Artifacts services" and "Version Control and Build services" which may be used to manage artifacts created during the product development process while maintaining the evolution of projects, code, and instances within the system.

[0117] As shown in Figure 3, the architecture of the DE platform 300 may also include a customer environment 310 with a “trusted true source” 312, customer tools 314, and an optional DE platform exclave 316. The customer environment 310 is the environment in which customer data resides and is processed by the DE platform 300 in a zero-trust manner. As previously mentioned, the DE platform enclave 302 provides a robust and scalable environment for securely handling critical workloads according to customer-specific needs by focusing on both zero-trust principles and characteristics such as hyperscale. In some examples, the DE platform exclave 316 may be located within the customer environment 310 to assist customer(s) 306 with their DE tasks and operations, including model splicing and digital threading.

[0118] If a customer 306 (e.g., user 204) intends to perform DE tasks using a DE platform 300 (e.g., IDEP 100), typical operations may include secure data ingestion and controlled data retrieval. Derived data generated through DE operations, such as revisions to updated digital model files or digital model parameters, may be stored only within the customer environment 310, and the DE platform 300 may provide tools to access metadata of the derived data. Here, metadata refers to data that can be viewed without opening the original data and may include version control information, timestamps, access control characteristics, etc. An exemplary embodiment may include secure data ingestion, which leverages zero-trust principles to ensure that customer data is securely uploaded to the customer environment 310 through a pre-verified secure tunnel, such as a Secure Sockets Layer (SSL) tunnel. This may enable direct and secure file transfer to designated cloud storage, such as a Simple Storage Service (S3) bucket, within the customer environment 310. An exemplary embodiment may also include controlled data retrieval, in which a temporary, pre-authenticated URL generated by a secure token-based mechanism is used for controlled data access, thereby minimizing the risk of unauthorized interaction. An exemplary embodiment may also include immutable derived data, in which transformed data generated through operations such as data extraction is securely stored within the customer environment 310 in compliance with zero-trust security protocols. An exemplary embodiment may also include a tokenization utility, in which a dedicated DE platform tool called a “tokenizer” is deployed within the customer environment 310 for the secure management of derived metadata in accordance with zero-trust guidelines.

[0119] The customer environment 310 may interact with other elements of the secure DE platform 300 and includes several features that handle data storage and secure interaction with the platform 300. For example, one element of the customer environment 310 is the “Trusted True Source” 312, which is the primary repository of customer data and guarantees data integrity and accuracy. Nested within this is the “Customer Bucket,” where data is securely stored with strict access control and data access is restricted to authorized users or processes through pre-authenticated URL links. This setup provides seamless interaction with other elements of the DE platform 300 while ensuring uncompromising data security within the customer environment 310.

[0120] The customer environment 310 may also include additional software tools, such as customer tools 314, which can be used based on specific customer requirements. For example, the “DE Tool Host” component may handle the DE applications necessary to manipulate customer data. It may include a DE Tool Command Line Interface (DET CLI) that enables user-friendly command-line operation of DE tools (e.g., DE Tool 102). The “DE Platform Agent” ensures smooth communication and management between the customer environment 310 and the elements of the DE Platform 300. Furthermore, there may be another set of optional DE tools designed to support customer-specific DE workflows. Native DE tools are typically restricted in access by proprietary licenses and end-user license agreements for which the customer pays. IDEP platform functions call native DE tools running within the customer environment 310, thereby strictly adhering to the zero-trust principle of the system design. Exemplary DE tools include, but are not limited to, proprietary and open-source versions of model-based systems engineering (MBSE) tools, augmented reality (AR) tools, computer-aided design (CAD) tools, data analysis tools, modeling and simulation (M&S) tools, product lifecycle management (PLM) tools, multi-attribute transaction space tools, simulation engines, requirements modeling tools, electronic modeling tools, test plan modeling tools, cost modeling tools, scheduling modeling tools, supply chain modeling tools, manufacturing modeling tools, cybersecurity modeling tools, or mission effectiveness modeling tools.

[0121] In some cases, an optional "IDEP Exclaver" 316 may be deployed within the customer environment 310 to support the customer's DE tasks and operations, oversee data processing, and provide hyperscale-like platform performance while strictly adhering to zero-trust principles. The IDEP Exclaver 316 is maintained by IDEP to run DE tools for customers requiring such services. The IDEP Exclaver 316 may include a "DE Tool Host" that runs the DE tools and a "DE Platform Agent" necessary for their operation. In this case as well, the native DE tools are typically restricted in access by proprietary licenses and end-user license agreements for which the customer pays. The IDEP Exclaver 316 utility manages proprietary DE tools hosted in the customer environment 310, for example, to implement model splicing and digital threading capabilities.

[0122] In some embodiments, the machine learning (ML) models and artificial intelligence (AI)-assisted approaches described herein are adapted to suit different customer instances of IDEP (see Figure 4) and the availability of training data. In one example, a pre-trained ML or AI model (e.g., within IDEP Enclave 302) is deployed when there are limitations on sharing customer data. In another example, the AI ​​model is deployed in a federated manner adjacent to DE agents and DE tools in the customer environment (e.g., within IDEP Exclave 316). In yet another example, the AI ​​model deployed within the customer environment is trained behind its firewall. In yet another example, the customer may enable the sharing of a subset of metadata about the training database located within the IDEP Enclave.

[0123] IDEP Deployment Scenario Figure 4 illustrates potential scenarios for instantiating an IDEP connected to a customer's physical system and IT environment according to several embodiments of the present invention. Specifically, Figure 4 illustrates various potential configurations for instantiating or instantiating an IDEP ("DE Platform") 402 connected to a customer's IT environment and physical system 404. The IT environment may reside on a firewall-protected virtual private cloud (VPC). The physical system may refer to a physical twin, as described with reference to Figure 1. In some embodiments, IDEP 402 may be instantiated as an enclave, such as 302 shown in Figure 3. For example, IDEP 402 may be instantiated on the cloud, perhaps in a Software as a Service (SaaS) configuration. The platform instance in these embodiments includes software and algorithms and may be described as follows:

[0124] 1. External Platform Instance 410: This option presents the IDEP as a separate platform instance. The platform interacts with the physical system through the customer's virtual environment or the customer's virtual private cloud ("Customer VPC"), which is connected to the physical system. 2. External Platform Instance 420 with Internal Agent: The IDEP is instantiated as a separate platform connected to an internal agent ("DE Agent") fully instantiated within the customer's VPC. For example, the IDEP may be instantiated as enclave 302, and the DE Agent may be instantiated as exclave 316 in the customer's VPC linked to a physical system. 3. External Platform Instance 430 with Internal Agent and Edge Computing: This scenario views the IDEP as a separate instance, connected to an internal DE agent fully instantiated within the customer's VPC, with the internal DE agent further linked to edge instances on the physical system ("DE Edge Instances"). The DE agents are nested within the customer's environment with smaller edge computing instances attached to the physical system. 4. Edge Instance Connection 440: This option indicates a DE platform directly linked to a DE edge instance on a physical system. The DE platform and physical system are shown separately, connected by an intermediate edge computing instance, illustrating the data flow. 5. Direct API Connection 450: This deployment scenario illustrates a DE platform that connects directly to a physical system via API calls. In this depiction, arrows extend directly from the platform's domain to the physical system's domain, signifying direct interaction through the API. 6. Air-Gapped Platform Instance 460: This scenario demonstrates that IDEP, as a DE agent, is fully instantiated on an air-gapped or isolated physical system. The platform operates independently of any network or internet connectivity, providing an additional layer of security by eliminating external access points and potential threats. Interaction with the platform in this context occurs directly on the physical system, and any data exchange outside the physical system is controlled according to strict security protocols to maintain the air-gapped environment.

[0125] Throughout these deployment scenarios, IDEP plays a crucial role in bridging the gap between the digital twin (DTw) established through IDEP and its physical counterpart. Regardless of how IDEP is instantiated, IDEP interacts with physical systems directly or through the customer's virtual environment. The use of edge computing instances in some scenarios demonstrates the need for localized data processing and the trade-off between real-time analytics and more precise insights in digital-physical system management. Furthermore, the platform's ability to connect directly to physical systems via API calls highlights the importance of interoperability in facilitating efficient data exchange between the digital and physical worlds. In all cases, the DE platform operates with robust security measures in place.

[0126] In some embodiments, IDEP deployments for the same physical system may include combinations of the deployment scenarios described above. For example, for the same customer, some physical systems may have direct API connectivity to the DE platform (Scenario 5), while others may have edge instance connectivity (Scenario 4).

[0127] Multimodal User Interface Figure 5 illustrates the use of a multimodal user interface 590 for an interconnected DE platform capable of handling various input and output modalities, including virtual reality (VR), mixed reality (MR), auditory, text, and code. These interfaces are designed to manage the complexity of data streams and decision-making processes, and to provide decision support, including option visualization, impact prediction, and invocation of specific decisions. Specifically, data streams 502 and 504 are processed by the analysis and control plane (ACP) 150 in Figure 1. The user interface can receive data streams from physical feedback loops 102 and virtual feedback loops 104, as well as from external expert feedback 114, analysis module 154, and a twin configuration set 156 of the ACP 150.

[0128] The multimodal interface shown in Figure 5 is configured to perform all DE tasks and actions described in the context of Figure 1 by accommodating both humans and bots / algorithms, and by handling complex issues such as the frequency and complexity of data streams, the timescale of decision-making, and the impact of latency. For human decision-makers, the user interface may need to manage inputs and outputs, while for algorithmic decision-making, the user interface may need to present rationale and decision analysis to the human user. Some examples of human interfaces include a dashboard interface 594, a workflow-based interface 596, a conversational interface 598, a spatial computer interface 592, and a code interface 599.

[0129] The dashboard interface 594 provides a customizable overview of data visualizations, performance metrics, and system status indicators. This enables monitoring of relevant information, partial review of documents, and decision-making based on dynamic data updates and external feedback. Such interfaces may be accessible via web browsers and standalone applications on various devices.

[0130] The workflow-based interface 596 guides the user through the decision-making process, presentation of relevant data, options, and contextual information at each stage. It integrates external feedback and is designed as a progressive web or mobile app. In the context of alternative tool selection, the workflow-based interface 596 may provide options on individual tools at each stage, or it may provide combinations of tool selections across various stages to achieve better accuracy or efficiency for the overall workflow.

[0131] The conversational interface 598 converts various input formats, such as text, prompts, voice, and audiovisual, into input text, which is then integrated into the DE platform workflow. Output from the DE platform may undergo the reverse process. This enables interoperability with the DE platform and, specifically, the manipulation of model splices. In the broader context of audiovisual input, the conversational interface may include data acoustics, which involves using sound to represent data, information, or events, and using auditory cues or patterns to communicate critical information to users, operators, or reviewers. Acoustic alerts (e.g., alerts transmitted via speakers, e.g., by voice) are particularly useful when individuals need to process information quickly without needing to visually concentrate on a screen. For example, acoustic alerts can be used to notify security analysts of potential threats or breaches.

[0132] Figure 5 also illustrates the use of the spatial computing interface 592 and the code interface 599 in the management of DTw and PTw. The spatial computing interface enables a more immersive and intuitive user experience and allows for real-time synchronization between DTw and PTw. The code interface allows bots and digital engineers to interact with the DE platform through scripts and code. The code interface also allows for the collection of user preferences, task history, and tool usage patterns for the purpose of selecting alternative tools.

[0133] Digital threads and autonomous data linkages As mentioned above, a "digital thread" is intended to connect two or more digital engineering (DE) models for traceability throughout the entire system engineering lifecycle and for collaboration and sharing among individuals performing DE tasks. In a digital thread, appropriate outputs from a preceding digital model may be provided as inputs to a subsequent digital model, thereby enabling information and process flow. In other words, a digital thread can be seen as a communication framework or data-driven architecture that connects traditionally siloed elements to enable the flow of information and actions between digital models.

[0134] Figure 6 illustrates the architecture and inherent complexity of digital threads by example disclosed herein. Specifically, Figure 6 is a schematic diagram comparing exemplary digital threads 600 of various complexityes for manipulating and / or connecting DE models by several embodiments of the present invention. In its most basic sense, a digital thread may “thread” DE models into a simple daisy-chain architecture 602, where any modification of an upstream DE model affects all downstream DE models of the modified DE model. For example, any modification of a parameter or process in DE model B causes a change in DE model C, which in turn causes a change in DE model D. Changes in causality thus cascade downstream. As another example, Figure 604 represents a more complex digital thread where a change in one DE model may affect one or more downstream models. In both 602 and 604, the digital threads are represented by a directed acyclic graph (DAG).

[0135] DAGs are frequently used in many types of data processing and structuring tasks, such as task scheduling and data compression algorithms. In the context of service platform and network complexity, DAGs may be used to represent relationships between different components or services within a platform. In digital threads 604, different models may depend on each other in different ways. Model A may influence models B, C, and D, models B and C may influence model E, and models D and E may influence model G. Such dependencies can be represented as a DAG, where each node is associated with a component (e.g., a model) and each directed edge represents a dependency.

[0136] A major problem with dealing with interdependent DE models is that the consistency of the graph can be polynomial in complexity, and potentially exponential. Therefore, if a node fails (e.g., the model becomes unreliable), this can cascade to the rest of the digital thread, potentially disrupting the entire design. Furthermore, adding nodes or dependencies to the graph does not result in a linear increase in complexity due to interdependencies between models. If a new model is added that affects or depends on several existing models, the resulting increase in graph complexity is essentially multiplicative, and therefore potentially exponential. The multiplicative nature of digital thread consistency is exacerbated by the enormous number of interconnected models, which can reach hundreds or even thousands. Diagram 606 is a partial representation of a real-world digital thread, illustrating the complexity of a digital thread and its multiplicative growth.

[0137] Figure 6 further illustrates specific cases 603, 605, 607, 608, and 609 of exemplary simple digital threads. Diagram 607 represents a degenerate digital thread where data is shared from a single DE model. Diagram 608 represents a model-versus-document digital thread, where data extracted from a single DE model (e.g., system attributes, performance attributes) can be used to generate or update a text-based document (e.g., a Capability Development Document (CDD)). Diagrams 603 and 605, generalized from 608, represent cases where data extracted from a single model can be used to update multiple models, and vice versa. Specifically, diagram 605 may represent the dynamic updating of a live document or magic document as described in the context of Figure 1. Here, the logic connecting the DE models shown is very simple, with data extracted from multiple DE models A, B, and C to update document model D. There is no interaction between the extracted data. Furthermore, diagram 609 illustrates a special case of digital threads where data is loaded into and extracted from a single model A. For example, as discussed below in the context of Figure 7, the input splice function of model A shown in 609 may be executed to update the model, and the output splice function of model A shown in 609 may be executed to produce a digital artifact for sharing. For these special simple threads, IDEP may provide the user with a GUI-based interface to connect the models and execute the digital threads. For complex threads like those in 606, a code-based interface may be required.

[0138] Model splicing for digital threading and digital twin generation As disclosed herein, model splicing encapsulates and partitions digital engineering (DE) model data and model data manipulation and access functions. Therefore, model splicing provides selective access to model data within a DE model file without exposing the entire DE model file, using access control to the encapsulated model data based on user access rights. Model splicing also provides DE models with a common, externally accessible application programming interface (API) for programmatically executing the DE model. Model splices thus generated can be shared, executed, modified, or further spliced ​​independently of the native DE tools and development platforms used to generate the input digital models. Standardization of DE model data, as well as generalization of API interfaces and functions, enables access to DE model type files outside of native software environments and allows linking of different DE model type files that might previously be incompatible. Model splicing further enables the scripting and coding of DE behavior, encompassing heterogeneous DE tools within a corpus of prescriptive program code, and facilitates the generation and training of artificial intelligence (AI) and machine learning (ML) models for the purpose of manipulating DE models through various DE tools across different stages of the DE process, DE workflow, or DE lifecycle.

[0139] Digital threads are created through user-instructed and / or autonomous linking of model splices. Digital threads are intended to connect two or more DE models for traceability throughout the entire system engineering lifecycle, as well as for collaboration and sharing among individuals performing DE tasks. In a digital thread, appropriate outputs from a preceding digital model are provided as inputs to a subsequent digital model, enabling information flow. In other words, a digital thread can be seen as a communication framework or data-driven architecture that connects traditionally siloed elements, enabling the flow of information between digital models. The extensibility of model splicing across many different types of DE models and DE tools allows for the scaling and generalization of digital threads to represent each and every stage of the DE lifecycle.

[0140] A digital twin (DTw) is a real-time virtual replica of a physical object or system with a bidirectional flow of information between a virtual domain and a physical domain, enabling monitoring, analysis, and optimization. Model splicing allows individual DE model files to be created into executable splices that can be linked autonomously and securely, thereby enabling the management of numerous DE models as a unified digital thread. This capability extends to linking previously incompatible DE models to create a digital thread, receiving external performance and sensor data streams (e.g., aggregated data from DE models or linked data from physical sensor data), calibrating the digital twin using data streams from physical sensors outside the native DTw environment, and receiving expert feedback that provides opportunities to improve simulation and model parameters.

[0141] Unlike a digital twin (DTw), a virtual replica, or simulation, is a mathematical model that mimics real-world behavior to predict outcomes and test strategies. While a digital twin uses real-time data and has bidirectional communication, a simulation focuses on scenario analysis and outcome prediction. In other words, a DTw reflects the state of a physical system in time and space. A simulation is a series of actions performed on a digital model that reflect potential future states or outcomes that the digital model may undergo in the future. A simulation model is a DE model in the context of an IDEP, as disclosed herein.

[0142] When testing different designs, such as variations in wing length or chord dimensions, multiple DTws (sometimes numbered 100-1000) may be created to bridge the gap between the system's design specifications and real-world embodiments, enabling seamless updating and tracking of variations across a vast number of variables, as detailed in the context of Figure 1. For example, if three variations of a system are created, each will have its own DTw with specific measurements. These DTws may be accessed and updated via API function scripts, allowing for easy input of new measurements from physical parts during the manufacturing process. By autonomously linking with appropriate data, DTws may be updated to reflect actual measurements of the part, maintaining traceability and ensuring accurate data representation across hundreds or thousands of models.

[0143] Exemplary Model Splicing Setup Figure 7 is a schematic diagram showing an exemplary model splicing setup according to several embodiments of the present invention. Specifically, Figure 7 is a schematic diagram showing an example of embedded CAD model splicing.

[0144] In this disclosure, a “model splice,” “model wrapper,” or “model graft” of a given DE model file includes (1) DE model data or digital artifacts extracted or derived from the DE model file, including model metadata, and (2) locators or copies thereof for splice functions (e.g., API function scripts) applicable to the DE model data. A model splice may take the form of a digital file or a group of digital files. A locator refers to a link, address, pointer, index, access key, uniform resource locator (URL), or similar reference to the aforementioned DE digital artifacts and splice functions, which themselves may be stored in an access-controlled database, a cloud-based storage bucket, or other type of secure storage environment. Splice functions provide unified and standardized input and output API endpoints or SDK endpoints for accessing and manipulating the DE model data. The DE model data is model type specific, and the model splice is associated with model type specific input and output schemas. One or more different model splices may be generated from the same input DE model file, based on a particular user application under consideration and subject to data access restrictions. In some contexts, the shorter terms “splice,” “wrapper,” and / or “graft” are used to refer to spliced, wrapped, and / or grafted models.

[0145] Model splicing is the process of generating model splices from DE model files. Correspondingly, a model splicer is program code or an uncompiled script that performs model splicing of a DE model. A DE model splicer for a given DE model type, when applied to a specific DE model file of that DE model type, retrieves, extracts, or derives the DE model data associated with the DE model file, generates and / or encapsulates a splice function, and instantiates an API endpoint or SDK endpoint to the DE model according to its input / output schema. In some embodiments, a model splicer includes a collection of API function scripts that can be used as templates for generating DE model splices. "Model splicer generation" refers to the process of setting up a model splicer, which includes establishing a comprehensive framework or template from which individual model splices can be inferred.

[0146] Therefore, a DE model type-specific model splicer extracts or derives model data from a DE model file and / or stores such model data in a model type-specific data structure. The DE model splicer further generates or enumerates splice functions that can call native DE tools and API functions for the application to the DE model data. A DE model splice for a given user application contains or wraps DE model data and splice functions specific to the user application, allowing access to only a limited portion of the original DE model file and enabling modification of that limited portion for collaboration and sharing with stakeholders of the given user application.

[0147] Additionally, a document splicer is a specific type of DE model splicer specific to a document model. A “document” is an electronic file that provides information as a public record. Documents include human-readable files that can be read without specialized software, and machine-readable documents that can be viewed and manipulated by humans with the help of specialized software such as word processors and / or web services. Thus, a document may include natural language-based text and / or graphics that are directly readable by humans without requiring additional machine compilation, rendering, visualization, or interpretation. A “document splice,” “document model splice,” or “document wrapper” for a given user application may be generated by wrapping document data and splice functions (e.g., API function scripts) specific to that user application, thus revealing text at the component or part level (e.g., title, content table, chapter, section, paragraph) via an API endpoint or SDK endpoint, allowing access to portions of the original document or document template for collaboration and sharing with stakeholders of a given user application, enabling modification while minimizing manual referencing and human error.

[0148] In the CAD model splicing example shown in Figure 7, the CAD model file diesel-engine.prt704 proceeds through a model splicing process 710, which includes a data extraction step 720 and a splice function generation step 730. This input DE model 704 is in a file format specific to the particular DE tool, .prt. Data extraction is performed via a DE model crawling agent, implemented as a model crawling script within the model splicer, which crawls the input DE model file and distills the model data with metadata 722. Metadata is data that can be viewed without opening the entire input DE model file and may include entries such as file name, file size, file version, last modified date and time, and potential user input options identified from user input 706. Model data is extracted and / or derived from the input DE model and may include, but is not limited to, parts (e.g., propeller, engine cylinder, engine cap, engine radiator, etc.), solids, surfaces, polygon representations, and materials. When the model splicer crawls the model files, it determines how to organize and access the model data as essentially defined by the DE tool 702 used for splicing the DE model, and establishes a model data schema. This data schema describes the structure and format of the model data, and parts of it are used to translate into or create input / output API endpoints using the corresponding input / output schemas. In some embodiments, the model data with metadata 722 may be stored in access-restricted storage 726, such as a “customer bucket” 312 in the customer environment 310 in Figure 3, so that model splices such as 742, 744, and 746 can be generated on demand when the input DE model 704 is crawled.

[0149] The model splicer further generates splice functions (e.g., API function scripts) 732 from the native API 702 associated with the input CAD model. In this disclosure, “native” and “primal” refer to existing DE model files, functions, and API libraries associated with a particular third-party DE tool, including both proprietary and open-source options. The native API 702 may be provided by a proprietary or open-source DE tool. For example, the model splicer may generate an API function script that calls the native API of a native DE tool to execute functions such as HideParts(parts_list) and Generate2DView(). These model-type specific splice functions may also be stored in a splice function database 736, also for on-demand generation of individual model splices. A catalog or specification of splice functions provided by different model splices supported by IDEP, and orchestration scripts linking multiple model splices, constitute the platform API. This platform API is a common, universal, externally accessible platform interface that masks the native API 702 of any native DE tool integrated into IDEP, thus enabling engineers from different disciplines to interact with unfamiliar DE tools and allowing previously incompatible DE tools to freely interoperate.

[0150] Next, based on user input or a desired user application 706, one or more model splices or wrappers 742, 744, and 746 may be generated and wrapped with splice functions or API function scripts that can apply a subset or all of the model data required by the user application to the original input model and / or wrapped model data to perform the desired operations and complete the task requested by the user. In various embodiments, the model splice may take the form of a digital file or a group of digital files, and the model splice may include, in any combination or permutation, locators to the aforementioned DE digital artifacts and splice functions or copies thereof. Any number of model splices / wrappers may be generated by combining a selection of model data, such as 722, with an API function script, such as 732. Since the API function scripts provide unified and standardized input and output API endpoints for accessing and manipulating DE models and DE model data, such API handles or endpoints can be used to execute model splices and establish links with other model splices without directly calling native APIs. Such API endpoints may be formatted according to an input / output scheme that matches the DE model file and / or the DE tool being used, and may be accessed by orchestration scripts or platform applications that act on multiple DE models.

[0151] In some embodiments, when executed, API function scripts input to or output from a DE model or DE model splice. An "input" splice function, or "input node" such as 733, is a model modification script that enables updates or modifications to the input DE model. For example, a model update might include changes made to model parameters or model configurations via the input splice function. An "output" splice function, or "output node" 734, is a data / artifact extraction script that enables data extraction or derivation from the DE model via its model splice. API function scripts may invoke native API function calls of native DE tools. Artifacts are the results of execution from the output API function script within the model splice. Multiple artifacts may be generated from a single DE model or DE model splice. Artifacts may be stored in restricted access cloud storage 726, or other similar restricted access customer buckets.

[0152] One advantage of model splicing is its inherent minimal privileged access control capability for zero-trust implementations of IDEP, such as those disclosed herein. In the various deployment scenarios described with reference to Figure 4, and within the context of the IDEP implementation architecture described with reference to Figure 3, the original DE input model 704 and model data storage 726 may reside in a customer bucket 312 in the customer environment 310 in Figure 3. The splice function 732 stored in the database 736 calls the native API 702. The execution or invocation of the splice function 732 may depend on job-specific authentication or authorization via a proprietary license of the DE tool (e.g., residing in the customer environment 310 in Figure 3), and / or the information security clearance level of the requesting user. Thus, model splicing debunks monolithic access to the digital model type file as a whole file and instead provides specific access to a subset of functions, enabling limited, purposeful, and auditable interactions with a subset of model type files constructed from component parts or atomic units assembled into parts.

[0153] Digital Threading of DE Models via Model Splicing Figure 8 is a schematic diagram illustrating the digital threading of DE models via model splicing according to several embodiments of the present invention. The digital thread is intended to connect two or more DE models for traceability throughout the entire system engineering lifecycle, as well as for collaboration and sharing among individuals performing DE tasks.

[0154] Model splice linking generally refers to jointly accessing two or more DE model splices via an API endpoint or splice function. For example, data may be retrieved from one splice to update another (e.g., the input splice function of a first model splice calls the output splice function of a second model splice), data may be retrieved from both splices to generate a new output (e.g., output splice functions from both model splices are called), and data from a third splice may be used to update both the first and second splices (e.g., input splice functions from both model splices are called). In this disclosure, "model linking" and "model splice linking" can be used interchangeably because linked model splices are mapped to correspondingly linked DE models. Similarly, linking DE tools generally refers to jointly accessing two or more DE tools via model splices, where model splice functions encapsulating heterogeneous DE tool functionality may interact and call each other, or be jointly called by orchestration scripts to perform DE tasks.

[0155] Therefore, model splicing makes it possible to transform individual digital model files into model splices that can be linked autonomously and securely, enabling the management of numerous digital models as a unified digital thread written in scripts. Within the IDEP disclosed herein, a digital thread is a platform script that calls platform APIs to facilitate, manage, or orchestrate workflows through linked model splices. Model splice linking provides a communication framework or data-driven architecture that connects traditionally siloed elements, enabling the flow of information between digital models through corresponding model splices. The extensibility of model splicing across many different types of digital models allows for the scaling and generalization of digital threads that represent each and every stage of the DE lifecycle, instantiating and updating DTw as needed.

[0156] In the specific example shown in Figure 8, the orchestration script 894 is written in Python code and designed to interact via API endpoints such as 892 to determine whether the CAD model meets the total mass requirements. API endpoint 892 is an output splice function and is part of the platform API 890. The platform API 890 includes not only splice functions but also platform scripts or orchestration scripts themselves, such as 894.

[0157] Orchestration script 894 is divided into three main steps. 1. Retrieving data from CAD model splice: A POST request can be sent via the IDEP platform API to execute the computer-aided design (CAD) model splice 871. This model splice provides a uniform interface for modifying and retrieving information about the CAD model 881. Parameters of the CAD model, such as hole diameters, notch openings, and flange thicknesses, can be sent in the request and set via the input splice function. The total mass of the CAD model can be derived from the model parameters and retrieved via the output splice function. The response from the platform API includes the total mass of the CAD model 881 and the uniform resource identifier / locator (URL) of the CAD model. The response may further include a URL for an image of the CAD model. 2. Retrieving data from a SysML model splice: Another POST request may be sent via the IDEP platform API to execute a System Modeling Language (SysML) model splice 872. SysML is a general-purpose modeling language used in system engineering. The output function 892 of model splice 872 retrieves the total mass requirements of the system from the SysML model 882. The response from the platform API will include the total mass requirements of the system. 3. Align variables and check if requirements are met: Compare the total mass from CAD model 881 with the total mass requirement from SysML model 882. If the two values ​​are equal, a message is printed indicating that the CAD model is aligned with the requirements. Otherwise, a message is printed indicating that the CAD model is not aligned with the requirements.

[0158] In short, the orchestration script 894, which may be implemented in the application plane 160 of IDEP100 shown in Figure 1, links digital models 881 and 882 via model splice API calls. The orchestration script 894 is a scripted platform application that modifies the CAD model, obtains the total mass of the modified CAD model, obtains the total mass requirements from the SysML model, and compares the two values ​​to check if the CAD model meets the requirements. In some embodiments, the platform application within IDEP100 operates on one or more DE models by utilizing a set of functions.

[0159] Model Splice Plane Figure 9 is a schematic diagram illustrating the linking of DE model splices in a splice plane and a comparison of digital threading with and without model splicing, according to several embodiments of the present invention. The lower model plane 180 shows an example of current digital threading practice, where each small ellipse represents a DE model, and linking between any two DE models, such as models 982 and 984, requires their respective connections to a central platform 910, requiring potential additional linkages from any model to any other model. The central platform 910 includes program code that can interpret and manipulate the original DE models of distinct model types. For example, under the control of a subject expert, platform 910 may prepare data from digital model 982 in a format that can be accessed by digital model 984 via the native API of digital model 984, thus enabling modifications of digital model 982 to be propagated to digital model 984. Any feedback from digital model 984 to digital model 982 requires similar processing via platform 910 so that the data from digital model 984 is converted into a format that can be accessed by digital model 982 via its native API. This hub-and-spoke architecture 934 is not scalable to the enormous number of digital models (e.g., hundreds or thousands) contained within a typical large-scale DE project, because model updates and feedback are only possible through the central platform 910.

[0160] In contrast, when DE models are spliced, each original model is represented by a model splice that includes the relevant model data and unified and standardized API endpoints for input / output, as shown in the upper splice plane 170. The splices in splice plane 170 may be connected through scripts (e.g., Python scripts) that call API endpoints or API function scripts, and may follow a DAG architecture as illustrated with reference to Figures 1 and 6. Note that in Figure 1, only the set of generated splices is shown in splice plane 170, but in Figure 9, the scripts that link the model splices are also shown in the splice plane for illustrative purposes. Such scripts orchestrate the workflow through digital threads built on interconnected DE model splices, and are therefore referred to in this disclosure as orchestration scripts or platform scripts. Furthermore, while the splice plane 170 is shown in Figure 1 as part of IDEP 100 for illustrative purposes, it should be noted that in some embodiments, the splice plane 170 may be implemented inside the customer firewall and be part of the DE platform agent, as illustrated in the various deployment scenarios shown in Figure 4. That is, individual API function scripts generated via model splicing by the DE platform agent may be tailored to call proprietary tools that the customer can access in a private environment. A centralized platform 910 with unique access to all native tools associated with all individual digital models shown in Figure 9 is not required. Instead, the orchestration script calls a universal API function script, which may be implemented differently in different customer environments.

[0161] Therefore, model splicing enables model splices, such as model splice 972 from digital model 982 and model splice 974 from digital model 984, to access each other's data in a purposeful and direct manner, thus enabling the creation of model-based "digital meshes" 944 by platform scripts, which can be linked autonomously without input from subject matter experts.

[0162] An additional benefit of moving from the model plane 180 to the splice plane 170 is that the DE platform allows the creation of multiple splices for each native model (see, e.g., Figure 7), each having a different subset of model data and API endpoints tailored to the splice's target application. For example, a model splice could be used to generate multiple digital twins (DTws) that map the design of a physical product or object into a virtual space. Bidirectional data exchange between the physical object and its digital object twin enables testing, optimization, verification, and validation of the physical object in the virtual world by selecting the optimal combination of digital model configurations and / or architectures from the parallel digital twins built on the model splice, each potentially responding differently to the same feedback from the physical object.

[0163] The IDEP disclosed herein, supported by model splicing, digital threading, and digital twinning capabilities, connects DE models and DE tools to enable easy and secure collaboration of digital engineering data across engineering fields, tool vendors, networks, and model sources such as government and public agencies, special program offices, builders, small businesses, federally funded research and development centers (FFRDCs), and university-affiliated research centers (UARCs). An IDEP application example 950 is shown on the right side of Figure 9, illustrating how data from many different organizations can be integrated to enable cross-domain collaboration while maintaining data security, traceability, and auditability. Here, DE models from multiple vendors or component builders are spliced ​​or wrapped by the IDEP agent, and data artifacts are extracted with data protection. Transforming DE models into data artifacts enables cross-domain data transfer and protection of critical information. This allows model owners to retain complete control over their DE models using their existing security and IT stacks, continue using the DE tools best suited to their purpose, and preserve the same modeling schema / ontology / profile best suited to their purpose. IDEP transforms the DE model into a microservice, providing minimal privilege data bits that traverse to relevant stakeholders without the DE model ever leaving its home server, or without being replicated or delegated. IDEP also offers easy data access and digital threading options via a secure web application or secure API.

[0164] DAG representation of threaded tasks Model splicing provides a unified interface between DE models, enabling model and system updates to be represented by interconnected, pipelining DE tasks. Figure 10 shows an exemplary directed acyclic graph (DAG) representation 1000 of pipelining DE tasks related to digital threads according to several embodiments of the present invention. In diagram 1000, tasks executed through a digital thread orchestration script (e.g., 894) are structured as nodes in the DAG. Thus, actions are interconnected and executed in a pipeline linking DE model splices to ranges of corresponding parameter values. Digital threads can thus be created by establishing correct connections between any model splices of corresponding models at relevant endpoints via an interpretable DE platform script.

[0165] Referring to Figures 1 and 8, the DAGs of threaded tasks are constructed from digital threads and are part of the application plane 160 of the DE platform. Different DAGs can target different DE actions. For example, in Figure 1, constructing or updating DTw 122 within the virtual environment 120 has its own DAG 124. Model splicing transforms DE models into data structures accessible via APIs, thus enabling the use of software development tools ranging from simple Python scripts to complex DAGs to perform DE actions. Digital threads in model splicing eliminate the scalability issues of digital thread management and accelerate the digital design process, including design updates based on external feedback.

[0166] Following the above description of the basic elements and core aspects of IDEP disclosed herein, a digital threading system that enhances the functionality of IDEP with respect to digital thread generation will now be described in detail.

[0167] Overview of Digital Threads Digital transformation represents a rapidly expanding market characterized by robust profit margins. However, its growth is hampered by unique challenges. Specifically, the creation of digital twins through interconnected models and simulations, known as "digital threads," is hindered by issues such as vendor lock-in, high licensing costs, and technical debt. The technology landscape for digital transformation is well-funded and dynamic, encompassing a range of technologies from the Internet of Things (IoT) and cloud-to-edge computing to API-first and code-first hardware, as well as advanced large-scale language models and AI. This presents substantial opportunities to integrate models and simulations with these technologies, thereby simplifying the creation of digital threads. Digital twins are part of the broader concept of Industry 4.0, envisioned as interconnected models that not only simulate but also enhance our physical reality. Despite their potential, over 90% of digital transformation initiatives struggle to achieve success. When successful, they bring unparalleled capabilities for optimization and innovation to companies, particularly through the use of AI in complex industrial systems. This invention envisions a future where technological barriers to digital transformation are eliminated, and digitalization becomes an easily accessible commodity. By integrating models and simulations through digital threads, this invention envisions the creation of an industrial metaverse, the democratization of technological innovation, and the provision of rich data for AI to learn. Instead of the current labor-intensive process of creating digital threads, the new method enables their mass production. This includes an intelligent application layer between models and simulations, integrated with a digital engineering platform, thereby facilitating customization and maintaining ease of integration even as models evolve. Thus, the new method enables the mass production of digital threads.By building an application layer on top of the model splice, the platform provides ease of customization and integration, as well as a complete suite of applications from third-party developers. The ultimate goal is to create a digital engineering ecosystem similar to a software development stack, delivering significant customer value and a positive user experience.

[0168] An interconnected digital engineering and certification ecosystem is a computer-based system that can be used for various validation and certification, as well as other documentation purposes. Digital documentation management systems and methodologies are integrated within the digital engineering and certification ecosystem to assist in the preparation of engineering and certification documents and support the creation of appropriate documentation using appropriate data suitable for verification and validation (V&V) purposes. For example, the system integrates with computer-based systems for digital engineering and certification and includes security and access controls to protect templates and documents from unauthorized access or modification. In some embodiments, the system includes a user interface for selecting templates and inputting data into them, and machine learning algorithms to recommend templates and assist in document preparation. In alternative embodiments, APIs and software IDEs (such as VS Code) perform these tasks automatically. In some embodiments, the system may generate documents without a user interface by directly linking system data and using system APIs, software IDEs, and machine learning algorithms to assist in document preparation. The system tracks and communicates approval decisions throughout the entire authentication process and provides metrics for measuring the system's efficiency, accuracy, and user satisfaction. The system is scalable and flexible and can be customized to support different types of authentication or other documentation processes and user needs. The system may also utilize blockchain technology to enhance system security and transparency, and augmented reality or virtual reality technology to improve visualization and interaction with digital documents.

[0169] Digital documentation subsystems are designed to improve efficiency, reduce wasted time and effort, and eliminate the risks associated with manual documentation processes. Such systems can contribute to the improvement of digitally designed products. As industry becomes increasingly digitized, many processes throughout the entire product lifecycle are managed by computer systems enhanced with automation and artificial intelligence. Tools such as digital threads may be employed to connect phases of the product lifecycle using accepted and authorized data sources (e.g., requirements, system architecture, technical data packages, computer-aided design (CAD) models, and project tasks). Digital threads develop an integrated framework that enables efficient and effective monitoring and evaluation of the lifecycle by dynamically linking a wide variety of information systems and datasets across various domains of the product lifecycle (e.g., design, manufacturing, quality) without requiring one-to-one data mapping.

[0170] The interconnected digital engineering and authentication ecosystem links models and simulation tools to their associated requirements to meet validation and authentication objectives. AI assistance in creating digital threads that connect models and tools improves the scalability and versatility of model usage and reduces the need for specialized skills when managing multiple models. AI-assisted capabilities form the foundation of the digital engineering system and include scalable sharing of large libraries of models and versatile linking of different models into digital threads for authentication or validation purposes. Data streams serve as training data for fine-tuning AI algorithms that can assist users in creating digital threads. Training datasets can be scaled using synthetic data generation and customized to train enterprise-specific models for customers.

[0171] AI-assisted digital thread creation offers tangible benefits to users, including ease of use through integration with customizable AI assistance using AI-assisted digital thread creation and fine-tuned large language models (LLMs), improved interoperability between digital engineering tools and services through the ability to link and share models through a secure web application or collection of APIs, increased productivity through optimization of computing resources and analysis into model alignment or certification / lifecycle requirement steps, broader adoption among users with lower skill set standards using appropriate digital engineering tools, direct integration of relevant regulatory and certification standards, compliance, computation, and test data into digital engineering workflows, increased potential versatility of use through the ability to generate reports that can be presented to users in an easy-to-read format and may include recommendations for improvements to the user's product's digital prototype, and enhanced user interaction with the system through the ability to utilize AI-assisted digital engineering services such as AI-assisted project planners and AI-assisted design review plans.

[0172] In some embodiments, "AI assistance" includes workflow extension, optimization of digital engineering processes, and product compliance activities. Examples of such processes include (1) scalable sharing of models, (2) creation of digital threads, (3) AI-assisted documentation and digital thread recalibration, and (4) training data for different services and industries, generated by customizing the use of the platform. Scalable sharing of models employs AI-assisted script wrappers and customized AI architectures that can be scaled to various training data used to fine-tune LLMs. Digital thread creation may include an AI-assisted digital thread creation process, in which work planning is performed with user input data and AI-assisted training. AI-assisted documentation using digital thread recalibration may include AI-assisted digital thread / security audits (training data is endpoint metadata tracked with respect to security architecture), AI-assisted data cloaking, AI-assisted M&S optimization (e.g., interrupting and improving existing simulations if specifications are not currently met, linking one model to the next with digital threads, and checking whether requirements are passed), and examples of high-frequency digital engineering. Finally, training data for different services and industries, generated by customizing the use of the platform, can create equivalent “engineering pile” training datasets (e.g., alignment with horizontal digital engineering workflows or vertical industry-specific models). Versatile, customized training datasets can be generated through a fine-tuned hierarchical implementation of the LLM. Some embodiments include the ability to train models with encrypted data and the expansion of synthetic data creation with platform-specific and customer-specific training data.

[0173] Ultimately, all engineering will become AI-assisted coding, and digital threads for software code definition will make all of this possible.

[0174] AI-assisted multipurpose link for generating digital threads The following details the AI ​​assistance to the digital threading process, as described in the context of Figures 6, 8, and 10, within the context of IDEP.

[0175] AI-assisted digital engineering workflow Figure 11 shows an exemplary schematic diagram of data from a digital thread used to train an AI algorithm to support a user's workflow, according to several embodiments of the present invention.

[0176] Specifically, the "V" diagram or V-model of systems engineering is shown in Figure 11. The V-model is a framework used in systems engineering. It is a graphical representation that shows the relationship between development phases and corresponding verification and validation (V&V) activities. The left side of the V represents the development phases, and the right side of the V represents the V&V phases corresponding to each development phase. The V-model is a systematic and integrated approach to the design, development, operation, and maintenance of a system and may be adapted for different industries.

[0177] Digital engineering is an integrated digital approach to systems engineering. The digital thread, implemented according to embodiments of the present invention, connects traditionally siloed elements, enables a connected data flow throughout the phases of the V-model, and provides an integrated view of data across the entire product lifecycle. Furthermore, the digital thread enables traceability from one phase of the lifecycle back to the previous phase, facilitates the sharing of information across different stages of the V-model, and improves the efficiency and productivity of the entire systems development process.

[0178] Furthermore, data from software code definition digital threads can be used to train artificial intelligence (AI) algorithms and assist in the orchestration of digital engineering workflows. In particular, the following are typical stages of the systems engineering process: 1. Requirements Analysis: This step involves identifying and defining the system requirements. This includes understanding the needs, constraints, and objectives of stakeholders. 2. Concept Development: In this step, various concepts are generated and evaluated to determine the best solution to meet the requirements. 3. System Design: In this step, the selected concept is developed into a detailed system design, including the specifications of components and subsystems. 4. Implementation: This step includes building, assembling, and testing system components and subsystems to verify that they meet the design specifications. 5. Verification and Validation: This step includes testing the system to ensure that it meets the requirements and that the design is correct. 6. Operation and Maintenance: This step includes the continuous operation and maintenance of the system to ensure that the system continues to meet the needs of stakeholders. 7. Retirement: This step includes the decommissioning and disposal of the system when it is no longer needed or when it has reached the end of its lifecycle.

[0179] Figure 11 shows exemplary digital engineering tasks along the V model, including a set of definition and decomposition tasks 1111 (e.g., system engineering management plan 1113, operational concepts 1115, system-level requirements 1117, subsystem requirements such as high-level design 1119, and component detailed design 1121), and a set of integration and test tasks 1131 (e.g., component verification 1133, subsystem verification 1135, system verification 1137, system validation 1139, and delegated system operation and maintenance 1141). Coding and test implementation hardware and software 1151 can bridge the two sets of tasks.

[0180] Furthermore, Figure 11 shows the Observe (1163), Orient (1165), Decide (1167), and Act (1169) loop ("OODA loop") within the schematic diagram 1161 of the AI-assisted workflow. The OODA loop is a decision-making framework that describes a cycle of taking action based on observed information. It provides a strategic framework that helps stakeholders make effective decisions quickly. This AI-assisted workflow, combined with the use of software code-defined digital threads, provides options for both expert and non-expert users to interoperate DE models and DE tools with greater efficiency and enhanced productivity.

[0181] In the observation phase 1163, the AI ​​algorithm receives input 1171, such as modeling and simulation (M&S) data or user input, and processes this information to understand the current state of the environment or a specific M&S step. In the situational judgment phase 1165, the machine learning engine uses its knowledge, expressed in the form of the AI ​​algorithm 1173, to understand the meaning of the observed data and determine the current situation. The AI ​​algorithm includes the use of a generative LLM that has been trained on previous datasets 1175 and fine-tuned. In the decision-making phase 1167, based on the information gathered and the understanding of the situation, the AI ​​algorithm recommends options 1177 to the user about the best course of action (e.g., suggesting the next step, completing the task, or providing a status alert). "AI assistance" means that the user makes a decision 1179 whether or not to accept the recommendation. In the action phase 1169, a human user takes action, or the system automates the action based on the algorithm's output. Finally, the AI ​​algorithm continues to observe, judge, decide, and act as it interacts with its environment, repeating the process. While the OODA loop decision-making framework is sometimes used to describe a machine learning engine operating sequentially within a DE platform, it should be noted that it may not imply any additional structure or functionality.

[0182] Feedback data can arise from user input (explicitly or implicitly) and / or from system compilation and / or platform orchestration script execution.

[0183] Integration of AI-assisted digital threads within the digital engineering and certification ecosystem. Figure 12 shows an exemplary schematic diagram of an AI-assisted digital thread enabling various DE services according to several embodiments of the present invention. A combination of scalable model sharing, versatile model linking, and documentation enables a variety of DE services.

[0184] As previously mentioned with reference to Figure 2, the interconnected DE and authentication ecosystem may include a user device 1206A, API 1206B, or other similar human-to-machine communication interface or machine-to-machine communication interface operated by user 1204. The ecosystem may further include a computing and control system 1208 (hereinafter "computing system 1208") connected to and / or including a data storage unit 1218, an artificial intelligence (AI) engine 1220, and an application and service layer 1222. In some embodiments, data from multiple uses of the ecosystem (or a portion of the above data) may be aggregated to develop a training dataset. For example, usage records 1217 collected via computing system 1208 may be despecificated or anonymized before being added to the training set.

[0185] As previously mentioned with reference to Figure 11, a typical workflow executed by a digital thread may include an OODA loop of observation 1263, situation assessment 1265, decision-making 1267, and action 1269. Such a workflow takes user metadata 1251 and / or user input 1253 as input via user 1204, user device 1206A, or API 1206B. The workflow may also take input from various DE tools 1231 (e.g., M&S parameters 1233) and information from a repository of common V&V products 1241 (e.g., M&S metadata 1243). Within the situation assessment phase, an AI algorithm may evaluate the observed data, for example, using an LLM fine-tuned on training data. Within the decision-making phase, an action may be recommended via AI assistance, and the user may decide whether the recommended action should be taken. The workflow then operates to generate user action 1235 for DE tool 1231 and / or user action 1245 for V&V product 1241.

[0186] Embodiments of an AI-assisted digital thread include several components. The AI-assisted digital thread is integrated with a digital thread in a computer-based system for DE and authentication. Data sources, functions, and data formats link and share models, and link models to each other or for documentation purposes. Embodiments may include methods to ensure the accuracy and consistency of this data. Additionally, the system may implement quality control measures such as data validation and verification. Machine learning algorithms and techniques may be used to recommend scripts to the user, link models, perform M&S activities, or assist with document preparation. For example, the system may assist with document preparation by using predictive modeling and decision tree algorithms to provide suggestions for data fields and values ​​based on the user's previous input and the context of the entire document. Additionally, the system may use LLM to generate document text in a combination of semantic (e.g., variable-driven) and transformer-based methods. Security and access controls may be implemented to ensure that only authorized users can access and modify the models. This may include measures such as authentication, encryption, and role-based access control, which ensure that only authorized users can access the system to modify models, run simulations, or prepare documentation.

[0187] Embodiments may further include the design of the system's user interface and user experience, including methods for selecting models to share, selecting scripts, and linking them with models. Methods for monitoring and evaluating the system's performance, including metrics for measuring efficiency, accuracy, and user satisfaction, may also be considered. This may include metrics such as the time taken to share a model of a particular M&S activity for a particular V&V purpose, or the time taken to create a digital thread, the accuracy of the input data, and user satisfaction with the system. Embodiments may include blockchain technology to further enhance the system's security and transparency. Security architectures for the interconnected digital engineering and authentication ecosystem may be applied to secure users, models, and documentation. Finally, augmented reality or virtual reality technologies may improve interaction with the visualization and digital engineering system.

[0188] In some embodiments, nested digital threads may be employed. Exemplary embodiments are shown below. JSON definition script recipe: {top-level goal: { #potentially target certification reqs sub-assembly: { #an assembly to be considered (CAD) model-details: { #our splices that get specific data the values ​​pointing to other scripts / modules }}}

[0189] The above may include a pointer to a script that is itself a digital thread, but JSON can also be considered a digital thread.

[0190] Implementation of an AI-assisted digital thread Figure 13 shows a process flow 1300 for generating a software code definition digital thread according to several embodiments of the present invention. The system starts in step 1310. Next, in step 1320, the system trains a script generation machine learning (ML) model using a training dataset that includes a set of training triplets, each containing a sample intent input, a corresponding sample model representation set, and a corresponding sample platform orchestration script, with the sample platform orchestration script connecting the corresponding sample model representation set to perform the corresponding sample intent input.

[0191] An example of such a training dataset is illustrated with reference to Figure 20. The user may provide text input indicating the need to write a finite element analysis (FEA) report from a computer-aided design (CAD) model and an FEA simulation. The IDEP model may splice the input CAD and FEA models. The user may further provide an orchestration script to link the model splices using an API endpoint to generate the desired report. For example, the orchestration script may execute a CAD model splice function to generate an .msh file, execute an FEA model splice function to generate an .rst or .json file from the .msh file, or call an LLM-based generating AI model to generate an FEA analysis report from the .rst or .json file. This set of user intent input, model splices, and orchestration scripts to connect the model splices to achieve the user intent is logged in the AI ​​training database 2050. Training with such training datasets may involve identifying API endpoint usage patterns (e.g., sequences of API endpoint calls to properly link models) and / or intermediate digital artifacts passed from one model splice to another. In addition to training LLM-based generative AI models, the AI ​​training database 2050 may be used for fine-tuning and validation.

[0192] Next, in step 1330, the system receives a first model representation of the first engineering model. In step 1340, the system receives a second model representation of the second engineering model. A model representation refers to an embodiment of the engineering model in the form of a model file(s), a model splice disclosed in the context of Figures 7–9, or a collection of digital artifacts derived from the engineering model.

[0193] In step 1350, the system receives intent input. Such intent input describes which tasks are intended to be performed by a software-defined digital thread and what results are expected from the execution of the digital thread. For example, intent input may include user actions on the IDEP, user prompts, commands from existing software code-defined digital threads, and requests from software agents on the IDEP, where the software agent encompasses any high-level AI agent authorized to request actions on the IDEP.

[0194] Next, in step 1360, the system uses a script generation ML model to generate a platform orchestration script that connects the first model representation and the second model representation based on the intent input, and the platform orchestration script performs the intent input.

[0195] Finally, in step 1370, the system stores the platform orchestration script as a software code-defined digital thread. This completes process flow 1300 in 1380.

[0196] In some embodiments, the “software code definition thread” may be written from scratch by the user, ported from another source, and / or received from an AI script that runs an ML model, and then edited or fine-tuned by the SME user, or equivalent thereto.

[0197] In some embodiments, generating a software code-defined digital thread includes training a script-generating machine learning (ML) model using a training dataset comprising a set of training triplets each containing a sample intent input, a corresponding set of sample model representations, and a corresponding sample platform orchestration script, wherein the sample platform orchestration script connects the corresponding set of sample model representations to perform the corresponding sample intent input; receiving a first model representation of a first engineering model; receiving a second model representation of a second engineering model; receiving intent input; generating a platform orchestration script that connects the first and second model representations based on the intent input, wherein the platform orchestration script performs the intent input; and storing the platform orchestration script as a software code-defined digital thread.

[0198] In some embodiments, generating a software code definition digital thread further includes receiving feedback data on a platform orchestration script and training and / or fine-tuning a script generation ML model based on the feedback data.

[0199] In some embodiments, generating a software code definition digital thread further includes providing a user interface coding environment in an interconnected digital engineering platform (IDEP), receiving multiple user selections of a first engineering model and a second engineering model, wherein the first engineering model and the second engineering model are selected and received by the user, receiving multiple corresponding model representations from the first engineering model and the second engineering model, receiving user-defined code for a user-defined platform orchestration script, determining and / or receiving corresponding intent inputs, determining corresponding model representation endpoints used in the user-defined code from the user-defined platform orchestration script, recording the first and second engineering models, the first and second model representations, corresponding intent inputs, corresponding model representation endpoints, and the user-defined platform orchestration script in order to generate a training dataset, and storing a training dataset for training a script-generating ML model.

[0200] In some embodiments, “guess” and / or “receive” fall within the following terms: (1) the user explicitly provides the information required; (2) the IDEP platform infers the corresponding information based on the user action; (3) the IDEP platform infers the corresponding information based on the user action and may prompt the user for confirmation if the probability of certainty is below a threshold (e.g., if the user explicitly turns on advanced AI training mode); and (4) any equivalent process. “Guess” may further include the AI ​​agent automatically determining intent. “Guess” may refer to the user’s plain language intent (e.g., English) being mapped by the AI ​​to specific potential actions on the platform. AI inference of user intent includes mapping the probability of relevance of potential actions and then recommending the action that is most likely to be appropriate (or at least one whose probability exceeds a threshold).

[0201] In some embodiments, connecting a first model representation and a second model representation based on intent input includes linking a first endpoint of the first model representation and a second endpoint of the second model representation based on intent input.

[0202] In some embodiments, generating a software code definition digital thread further includes evaluating a first engineering model and a second engineering model within an interconnected digital engineering platform (IDEP) for sufficiency in performing intent inputs using an adequate machine learning (ML) model.

[0203] In some embodiments, generating a software code definition digital thread further includes, in response to sufficiency being determined, using a recommender ML model or a script generation machine learning (ML) model to determine a first representation endpoint in a first model representation related to an intent input.

[0204] In some embodiments, generating a software code definition digital thread further includes using a recommender ML model to determine the relationship between a first representation endpoint and a second representation endpoint based on intent input.

[0205] In some embodiments, the platform orchestration script includes script code that reads data from a first model representation and / or a second model representation.

[0206] In some embodiments, the platform orchestration script includes script code that writes data to a first model representation and / or a second model representation.

[0207] In some embodiments, the platform orchestration script includes scripted code for connecting inputs to a second model representation that is connected to the output of a first model representation.

[0208] In some embodiments, generating a software code definition digital thread further includes executing a platform orchestration script for a second model representation, wherein the output from the first model representation is the input for the second model representation.

[0209] In some embodiments, generating a software code-definition digital thread further includes reading data from a first model representation, performing calculations on the data, and writing the calculation results to the first and / or second model representations.

[0210] In some embodiments, generating a software code definition digital thread further includes receiving a third model representation of a third engineering model, which the platform orchestration script then further links with the first and / or second model representations.

[0211] In some embodiments, generating a software code definition digital thread further includes executing a platform orchestration script by calling one or more API or SDK endpoints associated with a first model representation and / or a second model representation.

[0212] In some embodiments, generating a software code definition digital thread further includes using an AI algorithm to determine a recommended third engineering model based on a first engineering model, a second engineering model, and a training dataset. The AI ​​model attempts to construct a DAG of the current digital thread and possible continuations. The same model responds whether it can link two models with applicable model splices to satisfy the user's intent, and in this case, also based on the user's intent, responds what the potential next model splices and related models in the thread are.

[0213] In some embodiments, the first engineering model and / or the second engineering model is a human-readable document file.

[0214] In some embodiments, one of the first or second models is a human-readable document file. Generating a software code definition digital thread further includes receiving a document template, analyzing the document template using an interconnected digital engineering platform (IDEP), determining the output data from the first and / or second model representations required to generate a document file using an AI algorithm, performing appropriate actions on the first and / or second model representations based on the requirements of the document template using a predetermined sequence to generate the output required for the document file, and generating the document file by assembling the document template and the outputs from the first and / or second model representations. The AI ​​algorithm may include (1) a recommender engine (content-based or collaborative filtering-based), (2) a neural network with additional capabilities such as Monte Carlo tree search that proposes parameter changes, (3) a transformer model for generating the document, and (4) a Markov decision process that determines state transitions based on the current state (e.g., analysis of the document template, verification of the model, understanding how to construct a new document based on the model type and other metadata). The recommender engine may determine possible document templates to suggest, or the type of data to incorporate for authentication, or other elements in the decision matrix. Neural networks may be supported by movements in MCTS-type search that recognize the best state, or by transformers that generate templates.

[0215] In some embodiments, generating a software code definition digital thread further includes predicting changes in the first model representation of a first engineering model and / or the second model representation of a second engineering model based on the changes in the first model representation and / or the second model representation.

[0216] In some embodiments, generating a software code definition digital thread further includes predicting changes in a first model representation of a first engineering model based on changes in a second model representation of a second engineering model.

[0217] In some embodiments, generating a software code definition digital thread further includes calling a second software code definition digital thread.

[0218] In some embodiments, one of the first engineering model and / or the second engineering model includes a neural network model. The model file may include a neural network model that interacts with one of the AI ​​models (e.g., an AI model that determines which inputs to receive from the model splice and / or writes orchestration scripts). The interrelationship may include training, providing training data, and extending the training data. Examples of neural network models that could be used instead of a DE model include (1) a neural network trained in the design space of a design that predicts drag and lift coefficients of design inputs (so the CAD model spliced ​​with the neural network generates drag and lift coefficients without running a simulation, although this can be time-consuming), and (2) a neural network trained on historical data for the engineering design toward final certification. This can be extended toward the entire certification process.

[0219] In some embodiments, generating a software code definition digital thread involves using a script generation ML model to generate a magic document associated with the software code definition digital thread, the magic document including an API endpoint to a human-readable text block, and the magic document is generated and updated in an audit log in response to the execution of at least a portion of a platform orchestration script using the API endpoint.

[0220] In some embodiments, the platform orchestration script includes a code block, which is associated with an information security tag, and the information security tag indicates a restriction on the execution of the code block.

[0221] In some embodiments, the first model representation of the first engineering model is a first model splice. Generating a software code definition digital thread means receiving a first engineering model file having a DE model type, the first engineering model file being in a native file format, extracting model data from the first engineering model file in the native file format, storing the model data in a model data storage area, and generating one or more externally accessible splice functions that enable external access to one or more digital artifacts derived from the model data stored in the model data storage area, the one or more externally accessible splice functions providing an addressable application programming interface (API) or software development kit (SDK) endpoint accessible to third-party applications and users, and the API endpoint or SDK endpoint The present invention further includes generating a first model splice of a first engineering model, which enables access to digital artifacts without accessing the entire first engineering model file and without requiring direct involvement by third-party applications and users using DE tools associated with the DE model type, wherein the first model splice includes access to a selective portion of one or more digital artifacts, and the first model splice includes access to at least one of one or more externally accessible splice functions, and the first model splice is accessible by third-party applications and users via an API endpoint or SDK endpoint, the API endpoint or SDK endpoint provides a unified programming interface to shareable model splices generated from DE models having DE model types.

[0222] In some embodiments, generating a software code-defined digital thread further includes training a script-generating machine learning (ML) model using a training dataset comprising a set of training triplets each containing a sample intent input, a corresponding set of sample model representations, and a corresponding sample platform orchestration script, wherein the sample platform orchestration script splices the models in the corresponding set of sample model representations to perform the corresponding sample intent input; receiving a first model representation of a first engineering model; receiving a second model representation of a second engineering model; receiving intent input; generating a platform orchestration script that splices the first and second model splices based on the intent input, wherein the platform orchestration script performs the intent input; and storing the platform orchestration script as a software code-defined digital thread.

[0223] Figure 14 shows a system for generating software code definition digital threads within a digital engineering system, according to an exemplary embodiment of the present invention. Specifically, Figure 14 provides an exemplary schematic representation of modules and data 1420 that can be used to generate a platform orchestration script 1452 by an interconnected digital engineering platform (IDEP) application 1480, according to an exemplary embodiment of the present invention.

[0224] The system may include access to at least one hardware processor 1494 responsible for executing program code 1492 in order to implement module 1420, which will be described later. The system may also include access to at least one non-temporary physical storage medium 1490, accessible by at least one hardware processor 1494, which stores the program code 1492 that can be executed by the hardware processor 1494. The program code may be stored and distributed among two or more non-temporary physical storage mediums and may be executed by two or more processors.

[0225] The system may include an IDEP application 1480 that controls a training module 1440 capable of training, fine-tuning, and / or validating an artificial intelligence (AI) module 1450. In one embodiment, the AI ​​module 1450 may include a script generation machine learning (ML) model. In another embodiment, the AI ​​module 1450 may include a splice / splicing recommender AI module.

[0226] To train the AI ​​module 1450, the training module 1440 may use training data 1442, which includes sample data triplets, each of which may include a sample intent input, a corresponding sample model file set, and a corresponding sample output platform orchestration script.

[0227] At runtime, user 1402 may provide intent input 1430 through user interface (UI) 1404. In one embodiment, intent input 1430 is a user prompt. More generally, intent input is a phrase or action encompassing the user's intent and may be generated from (1) user actions recorded by UI 1404, (2) explicit user prompts, (3) previously generated and stored software code definition digital threads, and / or (4) requests from software agents on an interconnected digital engineering platform (IDEP).

[0228] The IDEP application 1480 may also generate or identify two model splices, model A splice 1460 and model B splice 1470, each associated with a DE model (1410, 1412) related to the intent input 1430. Model A splice 1460 may include splice data 1462 and splice function 1464. Similarly, model B splice 1470 may include splice data 1472 and splice function 1474. Model splices 1460 and 1470, their data, and their functions are accessible through splicing APIs 1466 and 1476.

[0229] The trained AI module 1450 may receive a user intent input 1430, two model splices 1460 and 1470, and generate a platform orchestration script 1452. The output platform orchestration script 1452 may perform intent input 1430 and connect model A splice 1460 and model B splice 1470 based on intent input 1430.

[0230] The IDEP application 1480 may store the generated platform orchestration script 1452 as a software code definition digital thread.

[0231] Alternative Embodiments Embodiments applicable to fields other than DE In an alternative embodiment, digital engineering models can generally be viewed as models and true sources of information. For example, they may be viewed as decomposed true sources of information from the internet, leading to the next generation of “Internet 4.0,” and possess the characteristics described below.

[0232] While the present invention has been described in relation to digital engineering and engineering-related models, the invention also has applications to information and data sources outside of engineering. In some embodiments of the present invention, a digital engineering system includes text or data linked as a digital thread with authenticated references to various sources in fields other than engineering. Examples of such sources may include peer-reviewed academic journals, regulatory or legal documents, common V&V products, financial reports, newspaper articles from trusted news organizations, and other authoritative sources. Such embodiments present the user with the ability to track the source of specific document text and data and verify that it is authenticated from a trusted source.

[0233] Therefore, in some embodiments, the live document or magic document is a newspaper article, scientific paper, medical paper, financial report, engineering document, media document, legal document, online encyclopedia, political speech, congressional, federal, or other government report, or other document from other sources. In one embodiment, when the live document or magic document quotes someone or refers to statistics from a certain source, a software code-defined digital thread within the digital documentation system helps ensure the immutability, reliability, and consistency of the cited or referred source or data.

[0234] In an era of generative AI and frequent false information sources in newspapers, political speeches, social media, and broadcast media, the present invention helps readers (e.g., online experts or news and / or entertainment consumers) determine what is true and what is not, based on their own judgment of the credibility and / or reliability of the true sources being cited. The IDEP platform enables seamless citation of true sources and makes it easier for readers to cross-check, reducing the risk of false or misleading information gaining widespread support.

[0235] In one embodiment, references or links to live documents or magic documents to a trusted true information source at the base, e.g., the original scientific report cited by a newspaper article or a political speech, can be cross-checked by readers or by an AI agent with a true probability model.

[0236] In one embodiment, rather than only publishing the report (and optionally sample data) as is common currently, trusted true information sources such as scientific reports and papers publish the underlying model and metadata at the time of publication. As a result, the model splicer can interface with third-party models and data sources and can generate and authenticate live documentation or magic documentation related to those models.

[0237] For example, scientists may publish not only the published paper and some sample data, but also the model underlying the research. In one embodiment, this digital documentation system enables the publication of live or magic scientific papers on the Internet / Web, which are linked to the underlying models, simulations, and data published by the scientists. This enables the scientific community, the media, and the general public to quote, review, annotate, and seamlessly comment on scientific results while fully visualizing the underlying models used to arrive at the results. Further, any updates from new subsequent experiments may be pushed directly to the scientific models published by the scientists, and the digital documentation (e.g., scientific publications) associated with the scientific models can be automatically updated in real time.

[0238] One advantage of such a system is that scientists can publish preliminary research earlier than usual, rather than waiting for all experiments and analyses to be completed first. This will accelerate the dissemination of cutting-edge scientific information between both the research community and the broader general public. Furthermore, third parties such as the media and the general public (e.g., social media) can cite the underlying scientific papers and scientific models provided by scientists as reliable sources of true information. Such an ability also enables third parties to reproduce experiments, review data, and confirm scientific conclusions as a safeguard against or to mitigate errors, fraud, or statistically insignificant results.

[0239] In other embodiments, the scientific papers include medical-related articles, such as articles in medical journals, medical references, and / or other articles or documents containing medical information (e.g., newspaper articles containing medical information). The medical papers can cite underlying true sources of information considered reliable in the medical community, such as peer-reviewed medical papers containing underlying medical models or authoritative medical reference information sources.

[0240] In some embodiments, auditing of underlying sources and source data is possible. In some embodiments, the documents can be accessible and auditable in a zero-trust secure manner. In other embodiments, the documents are stored in a decentralized manner, for example, in a decentralized data store or a blockchain. In other embodiments, the documents are stored in a centralized manner, but the metadata associated with the documents and their sources can be stored in a decentralized manner, for example, in a decentralized data store or a blockchain.

[0241] In short, the software-defined digital threading of centralized and decentralized data sources brings possibilities for Web 4.0-type applications across various data sources available on the Internet / Web.

[0242] Generation of training data A software code-defined digital thread is a single program code or program script that connects data from two or more digital engineering models, data sources, or physical artifacts to accomplish a specific mission or business objective. In one embodiment, training data is generated for the software code-defined digital thread using a model splicer without AI assistance. The IDEP platform monitors and records user interactions to generate the training dataset. For example, a software developer using the IDEP platform generates a platform orchestration script from a model splice. The IDEP platform monitors the SME or software developer who generates the software code-defined digital thread by manually linking the model splices and manually writing the code, and then generates training data based on information inferred from the code.

[0243] In one embodiment, the relevant ML model (e.g., a digital thread generation model, an API endpoint recommendation model, etc.) can be trained through an IDEP-recorded feedback loop. In this embodiment, IDEP records platform API scripts, user input, documentation, and modifications and improvements to the DE model API endpoints, and uses these modifications to generate training data for the ML model.

[0244] IDEP platform with pre-trained ML models In one embodiment, training may be performed by an entity separate from IDEP. In such an embodiment, the relevant ML model may be pre-trained or trained separately, and the trained ML model may be stored, remembered, and / or transferred for use by the IDEP platform. In such an embodiment, the execution of the program code utilizes the pre-trained ML model without necessarily requiring further training steps. In one embodiment, a given ML model is pre-trained, and the IDEP system collects training data as described herein, and further trains, fine-tunes, and / or validates the given ML model.

[0245] Feedback data for improving digital threads When a digital thread is executed, feedback may be returned from the generated documentation or output (e.g., simulation results) to the input engineering model. For example, verification documentation may indicate that verification failed, and the system returns to modify the underlying engineering model to satisfy the requirements using either the SME input or an ML model trained on previous SME inputs. Thus, in one embodiment, the non-temporary physical storage medium for digital thread generation further includes program code for analyzing the generated software code-defined digital thread in relation to a first input engineering model, and for modifying the first input engineering model based on the output of the software code-defined digital thread.

[0246] In one embodiment, the analysis steps may be iterative, and the results of each iteration (e.g., running a digital thread, analyzing its output, modifying the input engineering model) are used as a starting point for the next iteration. This approach may be used to improve and refine a product or design by successively approximating the data and / or parameters of the engineering model underlying the design.

[0247] In another embodiment, once the digital thread computes an output, feedback regarding the result is sent to the first input engineering model. For example, while completing a requirements check of the design, the digital thread may indicate whether the design meets the requirements or not. The result is then recorded in the input engineering model(s).

[0248] In both examples, IDEP may be used to host the analysis and feedback cycles, respectively, suggesting that the parameters of the input engineering model(s) be stirred to test the range of design parameters that satisfy the design requirements. In one embodiment, the requirements check digital thread may be nested within the master analysis and feedback digital threads that run on IDEP.

[0249] In yet another embodiment, the script generation ML model may be further trained or fine-tuned using a feedback loop that includes input engineering models and their modified “output” versions. For example, once the script generation ML model generates a software code definition digital thread, the latter may be run once or multiple times with different input engineering models, leading to modifications of the input engineering models. This process results in one or more input / output engineering model pairs that can be used to further train or fine-tune the script generation ML model.

[0250] In yet another embodiment, a script generation ML model that generates a digital thread with API endpoint recommendations may be further trained or fine-tuned using a feedback loop that includes input engineering models and recommended API endpoints. For example, once a script generation ML model generates a software code definition digital thread, the latter may be run once or more times with different input engineering models, resulting in one or more recommended API endpoints. This process yields one or more sets of input engineering models and recommended API endpoints that can be used to further train or fine-tune the script generation ML model.

[0251] In another embodiment, a script generation ML model that generates a digital thread with output documents may be further trained or fine-tuned using a feedback loop that includes input engineering models and output documents. For example, once a script generation ML model generates a software code definition digital thread, the latter may be executed once or more times using different input engineering models, resulting in one or more output documents. This process yields one or more sets of input engineering models and output documents that can be used to further train or fine-tune the script generation ML model.

[0252] User input for generating platform orchestration scripts In one embodiment, the AI ​​algorithm further considers user input when generating orchestration scripts. Thus, in one embodiment, a non-temporary physical storage medium further includes program code for receiving user input, and one or more orchestration scripts are generated based on the user input.

[0253] In one embodiment, user input is selected from user actions and user prompts on the IDEP. In one embodiment, intent input is selected from a group consisting of user actions on the interconnected digital engineering platform, user prompts, existing software code definition digital threads, and requests from software agents on the interconnected digital engineering platform.

[0254] Intent input may also be a separate digital thread, as explained in the detailed disclosure.

[0255] Manual generation of script code for digital threads in software code definition. In one embodiment of the present invention, the platform orchestration script is generated by a software developer and / or SME, but without the use of AI automation. The user manually codes the digital threads using the IDEP platform and model splicer. In one embodiment, the user generates the code without AI assistance. In another embodiment, the user generates the code with some form of AI assistance. In yet another embodiment, the AI ​​generates the code, but the user must accept it before use. Other combinations and partial combinations of users working with AI-assisted agents are within the scope of the present invention.

[0256] Accordingly, one embodiment of the present invention is a non-temporary physical storage medium comprising program code for providing a user interface coding environment in an interconnected digital engineering platform (IDEP), receiving a plurality of user selections of a first engineering model and a second engineering model, wherein the first engineering model and the second engineering model are selected by the user, receiving a plurality of corresponding model representations from the first engineering model and the second engineering model, receiving user-defined code for a user-defined platform orchestration script, and determining and / or receiving corresponding intent input. User-defined code can be input into the user interface coding environment by the user by typing new code, copying and pasting and reusing code, generating template code with AI assistance, and by the user further editing and fine-tuning the code.

[0257] Software code definition without model splicing: digital threads One embodiment of the present invention includes a software code definition digital thread that does not use a model splicer. Various embodiments are considered herein. In one embodiment, the IDEP interfaces directly with a third-party tool using its own API and / or SDK and a native file format, without model splicing. In another embodiment, data is extracted from a native file format and stored in the IDEP platform, but without using a model splicer with a common external API. Other embodiments include using a model representation that is not a model splicer, as described in the detailed disclosure.

[0258] Digital threads for digital documentation In various embodiments, the generated software code-defined digital thread is configured to create and / or maintain a live DE document (i.e., a magic doc). In another embodiment, the generated software code-defined digital thread is configured to output a user-readable document.

[0259] In one embodiment, the generated software code-defined digital thread includes instructions to generate a document generation ML model that is fine-tuned for a specific document type, such as a specific document structure (e.g., a result report, an authentication report) or specific content (e.g., drone documentation). In this embodiment, the software code-defined digital thread includes instructions to assemble a training data set that includes a plurality of sample documents of the target document type, and instructions to train the ML model to generate documents of the target document type using a specified cost function at a predetermined maximum level. In one embodiment, the document type is a live DE document.

[0260] Single Model Orchestration Script and Recursive Digital Thread In one embodiment, the software code-defined digital thread includes a platform orchestration script for a single engineering model file. An example of a platform orchestration script with only one model file may include reading data from the single model file, performing calculations, displaying data, and / or inserting data into the model file. Thus, in one embodiment, the non-transitory physical storage medium may further include program code for executing at least one of one or more externally generally accessible splice functions of the model splice that accesses a selected portion of one or more digital artifacts from the model splice and performs at least one action or calculation thereon.

[0261] In some embodiments, intent input may be a command from another software code-defined digital thread or software agent. In other embodiments, the first software code-defined digital thread may include instructions to execute a second software code-defined digital thread. For example, a digital thread for generating live documentation about product attributes may include instructions to execute a digital thread for calculating the total mass of the product.

[0262] In one embodiment, the requirements check digital thread may be configured to iteratively or recursively traverse requirements and their variations.

[0263] AI-assisted application for DE tools Figure 15 shows an illustrative schematic diagram of a digital engineering tool applied to requirements files and design files according to several embodiments of the present invention. In one embodiment, a digital thread is generated in IDEP using user input from subject experts without AI assistance.

[0264] As previously mentioned with reference to Figures 2 and 12, the interconnected DE and authentication ecosystem may include a user device 1506A, API 1506B, or other similar human-to-machine communication interface or machine-to-machine communication interface operated by user 1504. The ecosystem may further include a computing and control system 1508 (hereinafter "Computing System 1508") connected to and / or including a data storage unit 1518, an artificial intelligence (AI) engine 1520, and an application and service layer 1522. In some embodiments, data from multiple uses of the ecosystem (or a portion of the above data) may be aggregated to develop a training dataset. For example, usage records 1517 collected via Computing System 1508 may be despecificated or anonymized before being added to the training set. As previously mentioned with reference to Figures 11 and 12, a typical workflow may take information as input from repositories of various DE tools 1531 and common V&V products 1541.

[0265] In the first step sequence, user 1504 uploads an MBSE file to the digital engineering platform 1551, and the digital engineering platform receives the MBSE file 1553. The platform then extracts requirements (e.g., weight) from the MBSE file 1555 and sends data based on these requirements to the MBSE tool, which is part of a set of digital engineering tools 1531 1557. After receiving the data 1559, the MBSE tool updates the data (e.g., weight) in the MBSE file 1561, exports the updated MBSE file to the digital engineering platform 1563, and the digital engineering platform receives the updated MBSE file 1565.

[0266] In a second step sequence that may be parallel to or consecutive to the first step sequence, user 1504 uploads a CAD file to the digital engineering platform 1571, and the digital engineering platform receives the CAD file 1573. The platform then calculates properties (e.g., mass) from the CAD file 1575 and sends the data 1577 to a CAD tool that is part of a set of digital engineering tools 1531. After receiving the data 1579, the CAD tool highlights any issues with the CAD file 1581 (which may include updating the CAD file), exports the updated CAD file to the digital engineering platform 1583, and the digital engineering platform receives the updated CAD file 1565.

[0267] Here, the digital engineering platform holds both the updated MBSE files from the first sequence and the updated CAD files from the second sequence.

[0268] AI-assisted scalable sharing of models through model splicing In one embodiment, "model splicing" and "model wrapping" enable the secure sharing of a digital model with a third party, as defined above. (The terms "splicing" and "wrapping" are used interchangeably.)

[0269] An AI-assisted approach to linking and sharing models in model-based systems engineering (MBSE) utilizes a combination of machine learning techniques to analyze and extract relevant information, suggest appropriate functions and parameters, create scripts to control digital engineering tools, and propose optimal script sequences for creating or modifying MBSE files. This enables dynamic changes to files and provides the option to incorporate user input to create variations of MBSE files. The machine learning engine is trained on datasets of user input and example scripts, enabling greater customizability and flexibility. This approach can be further enhanced by using finely tuned language models better suited to understanding the specific language and context of MBSE files. With its ability to continuously improve performance over time, the AI-assisted approach enables efficient and effective manipulation of MBSE files.

[0270] In some embodiments, MBSE files can be generalized to digital models. Digital models are machine-readable and often represent complex systems through models that are rendered via a UI for human interpretation (i.e., model-driven engineering (MDE)). The difference between MDE files and digital models is that digital models emphasize that the model remains machine-readable and can be manipulated by human users without a UI / UX.

[0271] Figure 16 shows an example of the steps for implementing scalable model sharing according to several embodiments of the present invention. The user uploads a file (e.g., MBSE), which is then received by a digital system, which then analyzes the file to extract relevant information. An AI algorithm suggests appropriate API functions and parameters for the file, creates a script to control the DE tool, and suggests a sequence of scripts. User input may create transformations of the file. The DE tool may be instructed to create or modify the file, thereby enabling the system to create functions that allow dynamic changes to the file. Finally, the system provides the user with a wrapper that enables model sandboxing.

[0272] Figure 16 outlines the process of uploading an MBSE file, extracting relevant information, suggesting appropriate functions and parameters, creating scripts to control digital engineering tools, creating or modifying an MBSE file, and enabling dynamic changes to the file. Additionally, it includes alternative options for incorporating user input to create variations of the MBSE file and outputting a wrapper as a sandbox model.

[0273] As previously mentioned with reference to Figures 2 and 12, the interconnected DE and authentication ecosystem may include a user device 1606A, API 1606B, or other similar human-to-machine communication interface or machine-to-machine communication interface operated by user 1604. The ecosystem may further include a computing and control system 1608 (hereinafter "Computing System 1608") connected to and / or including a data storage unit 1618, an artificial intelligence (AI) engine 1620, and an application and service layer 1622. In some embodiments, data from multiple uses of the ecosystem (or a portion of the above data) may be aggregated to develop a training dataset. For example, usage records 1617 collected via Computing System 1618 may be despecificated or anonymized before being added to the training set. As previously mentioned with reference to Figures 11 and 12, a typical workflow may take information as input from a repository 1641 of various DE tools 1631 and common V&V products.

[0274] In the first step sequence, user 1604 uploads an MBSE file to the digital engineering platform 1651, and the digital engineering platform receives the MBSE file 1653. Next, a machine learning engine on the digital engineering platform analyzes the MBSE file to extract relevant information 1655. Then, the machine learning engine (e.g., an AI algorithm) proposes appropriate API functions and parameters for the MBSE file 1657. Next, the machine learning engine creates a script to control the digital engineering tool 1659. Then, the system instructs the appropriate digital engineering tool 1667 to create or modify the MBSE file.

[0275] In an alternative sequence, the machine learning engine proposes a sequence of scripts 1661, and the user 1604 provides text input 1663. The user input may create a variation of the MBSE file 1665. The sequence then proceeds to step 1667 as described above.

[0276] After step 1667, the machine learning engine creates a function 1669 that allows for dynamic changes to the MBSE file. Finally, the system outputs a wrapper 1671 that enables sandboxing of the model.

[0277] AI-assisted functions are created by a machine learning engine using a combination of supervised and unsupervised learning techniques. Once trained, the model can be applied to MBSE files to suggest appropriate functions and parameters, create scripts to control digital engineering tools, and suggest sequences of scripts for optimal results. Additionally, the machine learning engine may also be trained on new data, improving its performance over time.

[0278] An example of an AI-assisted approach involves using a finely tuned language model. In this scenario, the machine learning engine is trained on a dataset of user inputs and example scripts based on MBSE files. The finely tuned language model is then better suited to understanding the specific language and context of the MBSE files, suggesting appropriate functions and parameters, creating scripts to control digital engineering tools, and suggesting sequences of scripts for optimal results. Additionally, as new data is added, the machine learning engine can continuously improve its performance over time. This approach allows for greater customization and flexibility, as the machine learning engine can be tailored to the user's specific needs and requirements.

[0279] AI-assisted multi-purpose linking of MBSE files Figure 17 shows an exemplary schematic diagram of AI-assisted multipurpose linking of MBSE files according to several embodiments of the present invention. In some embodiments, this embodiment extends or modifies the implementation steps of scalable sharing of models, as shown in Figure 7. The user uploads a file (e.g., MBSE), the file is then received by a digital system, and the digital system then analyzes the file to extract relevant information. An AI algorithm proposes appropriate API functions and parameters for the file and creates a set of wrappers for V&V purposes. Multiple wrappers are then linked into a digital thread. Corresponding DE tools may be connected to the wrappers, or the corresponding DE tools may be linked sequentially. Under this paradigm, a repository of common V&V products may monitor the digital thread, particularly after the corresponding DE tools have been linked sequentially. User input may refine and execute the digital thread. Note that the initiation step may be the same regardless of which model is being linked, and the system, based on user input, creates a link between two models in an AI-assisted manner, and the stepwise linking of models may result in a complete or partial digital thread.

[0280] The system receives files, analyzes them, and extracts relevant information. The AI ​​algorithm then suggests appropriate API functions and parameters to prepare the files. The platform generates API scripts using API calls from relevant tools, creating API endpoints that return relevant files (e.g., mesh files or analysis result files). This process may be repeated with additional files to create links between multiple files, and the machine learning engine can learn from the use of API endpoints to provide better recommendations for connecting digital engineering models in the future.

[0281] As previously mentioned with reference to Figures 2 and 12, the interconnected DE and authentication ecosystem may include a user device 1706A, API 1706B, or other similar human-to-machine communication interface or machine-to-machine communication interface operated by user 1704. The ecosystem may further include a computing and control system 1708 (hereinafter "Computing System 1708") connected to and / or including a data storage unit 1718, an artificial intelligence (AI) engine 1720, and an application and service layer 1722. In some embodiments, data from multiple uses of the ecosystem (or a portion of the above data) may be aggregated to develop a training dataset. For example, usage records 1717 collected via Computing System 1718 may be despecificated or anonymized before being added to the training set. As previously mentioned with reference to Figures 11 and 12, a typical workflow may take information as input from repositories of various DE tools 1731 and common V&V products 1741.

[0282] In the first step sequence, user 1704 uploads an MBSE file to the digital engineering platform 1751, and the digital engineering platform receives the MBSE file 1753. Next, a machine learning engine on the digital engineering platform analyzes the MBSE file to extract relevant information 1675. Then, the machine learning engine (e.g., an AI algorithm) suggests appropriate API functions and parameters for the MBSE file 1757. At this point, user 1704 may input input (e.g., search terms, feedback to system recommendations) (e.g., via text message) 1759. Then, the platform creates a set of wrappers for V&V purposes 1761 based on the user input and / or the suggested appropriate API functions and parameters.

[0283] After the set of wrappers is created, the platform assists in determining the arrangement of the wrappers 1767, and multiple wrappers are linked to the digital thread 1769. In parallel with or consecutively with steps 1767 and 1769, a corresponding set of DE tools is connected to the wrappers 1763. Then, based on the digital thread from step 1769 and the corresponding set of DE tools from 1763, the corresponding DE tools are linked in sequence 1765. At this point, the repository of common V&V products 1741 may monitor the digital thread 1771. The user may also input information 1773 to improve and / or run the digital thread.

[0284] Therefore, the platform generates API scripts using API calls from relevant tools and creates API endpoints that return relevant files (e.g., mesh files or analysis result files). Users may upload additional files, and the process is repeated to link one or more files. As a concrete example, consider a user providing text input or prompts to link a CAD file with an FEA tool. An AI algorithm creates the necessary wrappers based on the user input. Another AI algorithm links the CAD file and FEA model to each other by checking the script and calling the API endpoints. A machine learning engine logs the use of the API endpoints and learns the relationships between them. Additionally, the user may link the CAD and FEA models with the final documentation by merging files and calling all API endpoints. The machine learning engine uses this feedback loop to make better recommendations on which digital engineering models should be connected.

[0285] One specific example of AI-assisted functionality in a machine learning engine is when the engine uses a finely tuned language model, such as GPT-3, and uses user input along with example scripts based on MBSE files and their API scripts as training data. This process begins with collecting and preparing a large dataset of MBSE files and their corresponding API scripts to serve as training data for the finely tuned language model.

[0286] The language model is fine-tuned using a supervised learning approach, and the model is trained to predict appropriate API functions and parameters for a given MBSE file. The fine-tuned language model can then be used to generate API scripts for new MBSE files based on user input and existing MBSE files. Additionally, unsupervised learning techniques can be used to further improve the model's ability to suggest relevant API functions. Since unsupervised learning algorithms can learn to identify patterns and relationships between various MBSE files and their corresponding API scripts, the system can suggest new and useful API functions that have not been used before.

[0287] Once a finely tuned language model is trained and validated, it is integrated into the platform's workflow. As the machine learning engine receives feedback from users and logs the usage of API endpoints, the AI-assisted capabilities continue to improve over time. Furthermore, the finely tuned language model can be periodically retrained to incorporate new data and continuously improve its accuracy and performance.

[0288] AI-assisted digital thread generation Figures 18–20 illustrate schematic diagrams of various tasks performed by an AI-assisted digital thread, as disclosed herein. In particular, Figures 18–20 illustrate tasks performed to generate report documents dynamically linked to CAD and FEA models. The illustrated actions are performed by a user (204, 206A / B), an IDEP (208, 217, 220, 222), one or more DE tools (202), and one or more V&V products (210), as described in the context of Figure 2.

[0289] More specifically, Figure 18 illustrates an exemplary process for extracting DE model (CAD or FEA) data for sharing, according to several embodiments of the present invention. While CAD and FEA models are explicitly described in this example, other types of DE models may be processed in a similar manner.

[0290] The process of preparing a CAD model for sharing begins in step 1802 when the user uploads a CAD file (e.g., sld.prt) to IDEP. IDEP receives the CAD model in step 1810 and transfers it in step 1812 to an AI algorithm (e.g., a recommender ML model) configured to suggest appropriate API functions and parameters for the CAD file. Then, in step 1814, IDEP creates an appropriate API endpoint that returns a mesh file (e.g., in .msh format). In some embodiments, the corresponding DE tool (e.g., a CAD tool) generates the mesh file in step 1820 by executing an API script (e.g., in Python, C#) on the API endpoint. The linkable CAD model 1815 is now ready to be shared. It should be noted that this process of preparing a CAD model may also be viewed as AI-assisted CAD model splicing. The digital artifact (e.g., mesh file) is generated from a CAD model splice with a dedicated API endpoint in the form of a splice function for accessing or generating this digital artifact. The AI ​​algorithm used in step 1812 may recommend these dedicated API endpoints from an API function library for CAD models. In some embodiments, this AI algorithm may be trained using an AI training database 1830 which may include a sample DE model file, a corresponding sample output API endpoint, and optionally, a corresponding common V&V product that can enumerate specific types of digital artifacts required for a particular V&V purpose. The AI ​​training database 1830 may receive relevant training data from common V&V products available on the platform (e.g., design verification documents such as dynamic and static stress analyses 1840).

[0291] Using the same steps, the process of preparing an FEA model for sharing begins in step 1802, when the user uploads an FEA simulation project file (e.g., a mechanical project file, .wbpj) to IDEP. The AI-assisted system receives the FEA file, and the AI ​​algorithm suggests appropriate API functions and parameters for the FEA file. The platform can then create an API endpoint that accepts a mesh file (e.g., .msh) as described above and returns a text data file (e.g., .rst). In some embodiments, the corresponding DE tool (e.g., an FEA tool) executes the API script (e.g., in Python, C#) and generates output text data files (e.g., .rst, .json). Now, the output of the FEA simulation in the digital model 1815 is ready to be shared.

[0292] Figure 19 illustrates an exemplary process for generating Magic Dock-type documentation according to several embodiments of the present invention. The process for performing AI-assisted documentation for the current project begins in step 1902 when the user uploads an example FEA report or template FEA report (e.g., MS Word, PDF) to IDEP. Optionally, the user may update or modify this example report in step 1904; that is, the user may overwrite system text. For example, the user may indicate that an .rst or .json file from the FEA simulation output is available for use in subsequent steps. IDEP receives the FEA report in step 1910 and, in step 1912, transfers the FEA report to an AI algorithm (e.g., a recommender ML model) configured to suggest the appropriate DE model artifacts or DE model inputs necessary to generate a complete version of the report (e.g., an .rst file from the FEA simulation). Then, in step 1914, IDEP may generate an output report using a finely tuned LLM from the suggested DE model artifacts, for example, in the form of a Magic Dock. The Magic Dock may be built on top of a document splice and may include appropriate API endpoints (e.g., addressable references and functional access to individual sections of the document) that can connect to the DE model for live, dynamic updates in response to changes in the linked DE model. In some embodiments, in step 1914, a web page may be created for the generated FEA report to provide secure web-based access to its API endpoint. In some embodiments, in step 1920, the corresponding DE tool (e.g., an FEA tool) executes an API script (e.g., in Python, C#) on the API endpoint to generate the data required for the report generation step 1914 (e.g., a mesh file for a CAD model, or a .rst or .json file for an FEA model).Now we are ready to share output report 1915, which includes the appropriate API endpoints to link to the two DE models (CAD model and FEA model) for live-linked data.

[0293] In some embodiments, the process 1914 used to generate an output report including the necessary API endpoints may use an ML model or AI algorithm trained using a training database 1930, which may include a sample input data artifact, a corresponding sample output report, and optionally, a corresponding common V&V product that can enumerate specific types of digital artifacts required for a particular V&V purpose. The AI ​​training database 1930 may receive relevant training data from common V&V products available on the platform.

[0294] In other embodiments, the process for proposing the DE model artifacts necessary to generate the output report 1912 may also include an ML model trained using an AI training database 1930 containing sample input DE model files and corresponding sample output data artifacts. In one embodiment, the input of such an ML model may also include API endpoints generated for different DE model files in step 1814 of Figure 18.

[0295] Figure 20 illustrates the linking of CAD and FEA models 1815 with analysis documentation 1915 according to an exemplary embodiment of the present invention. This process begins in step 2040 when IDEP detects that the user is writing an orchestration script to merge files by calling API endpoints from all three (e.g., CAD API endpoint, FEA API endpoint, and document API endpoint). An example of an orchestration script for linking the models and documentation and generating a report is provided below. This software code defines a digital thread that generates an FEA report from digital artifacts extracted from the CAD and FEA models at runtime.

[0296] import IstariCollaborationPlatform # API endpoint for extracting .msh file from CAD file extract_api= “https: / / istari.com / api / enterprise / CustomerCo / models / CAD / extract / {unique_ID}” # API endpoint for performing FEA analysis on .msh file fea_api= “https: / / istari.com / api / enterprise / CustomerCo / models / FEA / analyze / {unique_ID}” # API endpoint for converting .rst file to PDF for FEA report pdf_api = “https: / / istari.com / api / enterprise / CustomerCo / models / report / pdf / {unique_ID}” # Unique ID for the CAD file unique_ID = “PI314159” # Call the CAD extraction API to get the .msh file response = requests.get(extract_api.format(unique_ID = unique_ID)) msh_file = response.content # Perform the FEA analysis on the .msh file by calling the FEA API response = requests.post(fea_api.format(unique_ID = unique_ID), data = msh_file) rst_file = response.content # Convert the .rst file into a PDF document for the FEA report headers = {“Content-Type: application / octet-stream”} response = requests.post(extract_api.format(unique_ID = unique_ID), data = rst_file, headers = headers) pdf_file = response.content # Save the PDF file to disk with open(“fea_report.pdf”, “wb”) as f: f.write(pdf_file)

[0297] During the execution of this digital thread, written as an orchestration script, the CAD tool may, in step 2080, execute a CAD model API script (e.g., the CAD tool is used while executing a requested API function) to generate an output file (e.g., .msh). The FEA tool may, in step 2084, execute an FEA model API script to generate an output file (e.g., .rst or .json). The LLM generative model may then, in step 2060, take data from the output file and generate a draft FEA analysis report. In step 2010, the user may edit the report and send the edited report or a link to the edited report elsewhere. Furthermore, the AI-assisted system may log the use of the API endpoints in step 2070 and learn that the .msh and .rst API endpoints are part of the "FEA report" and may be called sequentially in the digital thread because they are called in close relation to each other.

[0298] In some embodiments, instead of manually writing the script, the user may provide an optional prompt in step 2030. The user enters text (e.g., a search or prompt) instructing IDEP to write an FEA report from the CAD model and FEA simulation. The LLM generative model in step 2060 interprets the user intent input and creates and executes a script that links the API endpoints appropriately to write the FEA analysis report. In some embodiments, both the script and the report may be presented to the user, and the user may provide feedback to the AI ​​algorithm. For example, the user may correct the AI ​​algorithm or indicate that the script may retrieve a .rst or .json file from the output of the FEA simulation. In some embodiments, the user may request a different report (e.g., provide additional text input) or accept the report but edit parts of the report for a specific use case. Such edits may also be logged as training data for the AI ​​algorithm.

[0299] In yet another exemplary embodiment, one or more of the following steps may be performed when generating and executing a digital thread to create an FEA analysis report dynamically linked to a CAD model and an FEA model, according to some embodiments of the present invention.

[0300] 1. Preparing the CAD model for sharing a. A user uploads a CAD file (e.g., .sldprt) to IDEP, which may then provide an HTTP 202 response. b. IDEP receives the CAD file (HTTP206 response), analyzes it, and extracts relevant information. c. The first AI algorithm proposes appropriate API functions and parameters for the CAD file. d. IDEP generates API scripts (using a second AI algorithm to write Python or C# scripts using API calls from the CAD tool, for example). One of the scripts converts the CAD file into a mesh for FEA analysis (for example, as an "output" function). The CAD tool can execute the API script (Python or C#) and generate a mesh file (.msh). e. IDEP creates an API endpoint that returns a .msh file (for example, Istari / USer / CAD / EP8736181). 2. Preparing the FEA model for sharing, linked from the CAD model. a. The user uploads an FEA simulation project file (e.g., a mechanical project file (.wbpj)) (HTTP 202 response). b. IDEP receives and analyzes the FEA file (HTTP206 response) to extract relevant information. c. The first AI algorithm proposes appropriate API functions and parameters for the FEA file. d. IDEP generates API scripts (for example, using a second AI algorithm that writes Python or C# scripts using API calls from the FEA tool). One of the scripts accepts an .msh file and performs an FEA analysis using parameters associated with the FEA file. The FEA tool can execute the API script (Python or C#) and generate an output file (.rst or .json). e. IDEP creates an API endpoint that accepts an .msh file and returns an .rst file (e.g., Istari / USer / FEA / EP8736181). 3. AI-assisted documentation for the current project a. The user uploads an example of an FEA report (such as in MS Word or PDF). b. IDEP receives an example report document (such as a Word file), analyzes it, and extracts relevant information. c. The AI ​​algorithm may create a sub-LLM that is finely tuned for the document's structure and content (e.g., tokenization-embedding-transformer). d. The AI ​​algorithm proposes the necessary DE model inputs (e.g., .rst files from FEA analysis) to generate a complete version of the report from the example. e. Users may update or correct the AI ​​algorithm (for example, by showing that they can obtain a .rst or .json file from the output of an FEA simulation to generate a report). f. IDEP creates a webpage for the FEA report fine-tuning LLM and the API endpoint for the FEA report (for example, "https: / / istari.com / api / enterprise / aerospace / models / CAD / FEA / report / {.rst file}?format={pdf|word|html}"). 4. Link CAD models and FEA models using user scripts and collect AI training data. a. The user writes a script to merge files and calls both the CAD API endpoint, the FEA API endpoint, and the Document API endpoint. b. The CAD tool executes an API script to generate a mesh file (.msh). c. The FEA tool executes API scripts and generates output files (.rst or .json) from the mesh files. d. The system logs the use of API endpoints and learns that the .msh and .rst API endpoints are related because they are called closely together from user scripts. e. Users can use the .rst file for further analysis. f. Linking CAD models and FEA models with analysis documentation, using user scripts, and collecting AI training data. g. The user writes a script to merge files and calls all three API endpoints: the CAD API endpoint, the FEA API endpoint, and the Document API endpoint. The CAD tool executes an API script and generates a mesh file (.msh). i. The FEA tool executes API scripts and generates output files (.rst or .json) from the mesh files. j. The LLM generation model takes data from a .rst file and generates a draft FEA analysis report. k. The system logs the use of API endpoints and learns from user scripts that the .msh and .rst API endpoints are part of the "FEA report" and that they can be called sequentially in a digital thread because they are called in close relation to each other in user scripts. l. Users can edit reports and submit them to their team or share them via a link. Any user feedback on reports (e.g., requests for different reports, report edits) may be logged as AI training data for the LLM generative model. 5. Generate AI-assisted orchestration scripts to link CAD models, FEA models, and analysis documentation. a. Instead of providing an orchestration script as in step 4a or 5a above, the user provides text input (via search or prompt) indicating the need to link a CAD model to an FED model, or to write an FEA report from a CAD model and FEA simulation. This is user intent input. b. IDEP performs steps 1, 2, and 3 above to create CAD, FEA, and documentation API endpoints. The corresponding DE tools connect to the API endpoints. c. An AI algorithm trained on API endpoint usage patterns (e.g., logged in step 4d or 5e above) helps to sequence CAD models, FEA models, and documentation with the appropriate API endpoints to generate reports. The corresponding DE tools are also linked in sequence. This AI algorithm can be implemented as a script generation ML model that creates digital threads in the form of orchestration scripts, as in the example described in the context of Figure 20. d. Users can refine and execute digital threads to generate the desired report. e. Any user feedback used to improve the digital thread and edit the report may be logged as AI training data.

[0301] The AI ​​training data collected in steps 4d, 5e, and 6d can be used to train the AI ​​algorithms in steps 1c, 1d, 2c, 2d, 3d, 5d, and 6c described above. This feedback loop, which monitors the API endpoints regarding when and how they are called, allows the system to make better recommendations regarding which DE models should be connected to each other and which DE models should be connected to which reports.

[0302] Generation and updating of digital threads and associated magic documents In an interconnected development environment platform (IDEP), a digital thread represents a specified splice function, supported by an orchestration or co-script that associates the appropriate model splice, and code annotations to improve overall code understanding. In certain embodiments, artificial intelligence (AI) is employed to create digital threads in response to user input. Within these systems, AI-powered digital threads may be combined with a “magic document” that provides explainability and enables an audit trail of the digital thread. This “magic document” may be generated with the help of AI and describes the process by which the digital thread efficiently translates the user’s intent into an orchestration script containing the relevant model splice and splice function. Specifically, the magic document generated by the IDEP may describe an embodiment of the user’s intent in the digital thread and may include pseudocode, scripts, data fields, and natural language-based descriptions. When the digital thread and its accompanying orchestration script are executed to perform a DE task, the magic document may record the completion of the task for auditability. The digital thread may contain the orchestration script in sequence. One or more corresponding magic documents for a digital thread may, as needed, invoke a subset of data points and example orchestration scripts. For example, a companion magic dock for a given digital thread might include important data points (e.g., material strength, battery life) and examples of key orchestration scripts that relate to a user intent (e.g., "increase the drone's wingspan by 1%") and can be executed to achieve it. In some embodiments, the dynamic nature of a magic document with dynamic data links can be timestamped, digitally signed, and made static. That is, the magic dock can also be transformed into a document that reflects the behavior of a particular digital thread in the IDEP at a specific time in a particular user context, thereby satisfying auditability requirements.In another embodiment, such a static magic dock may be dynamically associated with a specific version of the DE task's certificate, even if it is no longer updated with the latest data.

[0303] Figure 21 shows swimlanes of an update process flow for a digital thread and associated magic document according to several embodiments of the present invention. Note that some steps and components in Figure 21 include numbers from two sets. The four-digit numbers (e.g., 2120, 2122) are reference characters, while the bolded one- and two-digit numbers (e.g., 1, 2) indicate the general chronological order of the steps.

[0304] The process for updating an existing digital thread 2104 involves at least two entities: a DE platform 2106 and a requester 2108. The process begins 2110 when the requester 2108 selects the digital thread and magic document required for updating 2112. The DE platform 2106 may then update the data fields of the linked information (e.g., user data, modeling and simulation parameters, simulation output) 2120. The DE platform 2106 may then determine 2122 whether the data matches predefined criteria. If the answer is "no", the requester 2108 may then manually update the digital thread and its associated magic document with text details 2114, referencing the data from the DE platform 2106. The requester 2108 may then update the metadata 2116, thereby completing the digital thread. Thus, the digital thread is ready for submission review and thus completes the first branch of the process 2118.

[0305] On the other hand, if the DE platform 2106 determines 2122 that the data actually matches a predefined criterion, the digital engineering platform 2106 may suggest 2124 relevant data fields available to the DE platform 2106 for the requester 2108 to include. The requester 2108 may then select or reject 2134 the suggested relevant data fields / recommended splices, splice functions, or digital scripts.

[0306] After the DE platform 2106 proposes 2124 the relevant data fields from the DE platform 2106 for the requester 2108 to include, the resulting data fields (e.g., user data prompts and digital thread field sections, including script and document fields) may be input 2126 into the NLP / LLM model. The NLP / LLM model may then generate the digital thread and companion magic document, or assist in the generation of the digital thread and magic document text by proposing scripts / sections and guiding the user. The DE platform 2106 may recommend 2130 the addition of code comments and script code to the digital thread, as well as the addition of text to the magic document. The requester 2108 may, as necessary, select or reject the recommended updates to the digital thread or document text. The requester 2108 may then update the metadata of the generated or modified orchestration script. This completes the digital thread and associated magic document, which are ready for submission review, and thus completes the second branch of the process.

[0307] For digital thread generation, as described above, an NLP / LLM model may be used. According to one embodiment of the present invention, the DE platform 2106 can train an NLP / LLM model 2136 by taking the outputs of two competing networks and evaluating them against a known standard. Training may also be performed using a generative adversarial network (GAN) trained to select from options in conjunction with Q-learning, in which case the agent is provided with reinforcement learning feedback and can operate with minimal supervision. Various other ML model architectures, such as those described in the following Machine Learning (ML) and Neural Networks sections, are within the scope of the present invention for performing digital thread generation.

[0308] In the embodiment of Figure 21, the inputs to the training process 2136 are represented using dashed arrows. These may include orchestration scripts and splices recommended by the system 2124, which have been selected or rejected by the user 2134 for the generation of digital threads. Other inputs to the training process 2136 may also include ML-generated 2128 digital thread updates, which have been selected or rejected by the user 2132. Such user feedback (e.g., 2132, 2134) may be stored in the training database on the IDEP for the purposes of ML training, fine-tuning, and testing / validation.

[0309] In some embodiments, the “User” and “Requester” are the same entity when performing the tasks of selection 2112, manual update 2114, feedback (2132, 2134), and / or metadata update 2116. In other embodiments, the Requester may perform only the initial selection task 2112 and then hand over the process to a second user to complete the tasks of manual update 2114 and metadata update 2116. In one embodiment, the metadata update 2116 is performed by a script that runs on the IDEP.

[0310] Figures 22 and 23 show detailed process flows for recommending, creating, and updating digital threads and magic documents according to several embodiments of the present invention. The detailed process flow for creating and updating digital threads and magnetic documents includes a first part (flowchart 2200 shown in Figure 22) and a second part (flowchart 2300 shown in Figure 23).

[0311] In the examples in Figures 22 and 23, the objective is to generate digital threads and associated documents (e.g., Companion Magic Dock) that may be useful for explainability and auditability purposes. The flowcharts shown in Figures 22 and 23 may utilize a generator ML engine or generator ML model trained to generate digital threads and associated documents from user input (e.g., user prompts) and, optionally, from document templates. The flowcharts shown in Figures 22 and 23 may also utilize a recommendation engine to recommend document templates, previous digital thread examples, or previous document examples to facilitate the digital thread generation process.

[0312] The process begins with user-initiated step 2204, which includes steps 2206 and 2208. In step 2206, the user creates a digital thread and magic document according to a template, based on the target result and purpose. Then, in step 2208, the user adds metadata tags related to specific purposes, priorities, etc. Next, system data step 2210 is executed, which includes steps 2212 and 2214. In step 2212, the documentation system adds any new digital threads and magic documents to the library along with their metadata. In step 2214, the system runs a script that periodically cleans and / or preprocesses the metadata, for example, so that features are on a corresponding common scale. Next, digital thread and magic document creation step 2216 is executed, which includes steps 2218, 2220, and 2222. In step 2218, the user selects specific metadata from a query menu. In step 2220, the system provides a list of templates or examples of previous digital threads or magic documents (including sanitized data) that contain the corresponding metadata. In step 2222, if a previous digital thread or magic document or template is unavailable, the system creates a new digital thread or magic document or template. Next, system output step 2224 is performed, including steps 2226 and 2228. In step 2226, the user selects a document appropriate for use for their chosen purpose and / or needs. In step 2228, the user may update the metadata of the selected document with new needs or requirements. The process may then return to step 2212 or proceed to step 2258 (point A), leading to flowchart 2300 (see Figure 23).

[0313] In step 2214, the system runs a script that periodically cleans and / or preprocesses the metadata, for example, so that the features are on a corresponding common scale. At this stage, the system may perform a clustering step 2254 as part of the AI-assisted documentation process 2252. The clustering step 2254 includes steps 2230, 2232, 2234, and 2236. In step 2230, the system applies supervised or unsupervised clustering techniques to create clusters in the library. In step 2232, the system implements silhouette coefficients or other performance metrics to measure the quality of the clusters in the template library. In step 2234, the system determines, based on the metrics, whether the clusters are high quality. If the clusters are not high quality (i.e., the result is "no"), the process returns to step 2230. If the clusters are high quality (i.e., the result is "yes"), in step 2236, the system creates smaller subgroups of the clusters in the document library (of templates and previous documents that are likely to be needed in combination).

[0314] Upon completion of the clustering phase 2254, the system proceeds to execute classification step 2256, which includes steps 2238, 2240, 2242, 2244, 2246, 2248, and 2250. In step 2238, the system uses and / or collects subgroup metadata as training data. In step 2240, the user provides text input (which may include specific data fields, metadata, or target results). In step 2242, the system runs a classifier algorithm to identify the subgroup that best fits the user input. In step 2244, the system recommends a template or previous document example accordingly. In step 2246, the user selects a template or rejects the recommended document example. The process can then proceed to fine-tune or further fine-tune the recommendation engine by returning to step 2242. In step 2248, if no previous digital thread or magic document or template is available, the system creates a new digital thread or magic document or template. In some embodiments, the system may recommend generic documents in steps 2246, 2248, and 2250, in which case the user may then select a new magic document from the recommended documents and subsequently build it. In step 2250, the user may update the metadata of the selected digital thread or magic document with new needs / requirements. In the embodiment of Figure 22, the clustering 2254 and classification steps 2256 together constitute AI-assisted document creation 2252 (using clustering and classifier algorithms). The process then may proceed to step 2258 (point A), leading to flowchart 2300 (see Figure 23).

[0315] Referring to Figure 23, flowchart 2300 continues flowchart 2200 in Figure 22 at point A, where point A connects steps 2258 and 2304. After step 2304, the system proceeds to step 2306, in which the user is assigned a digital thread or magic document to update. After step 2306, the process may proceed to semi-automatic digital thread or magic document update 2308, or AI-assisted digital thread and magic document update 2334.

[0316] The semi-automatic digital thread or magic document update step 2308 includes steps 2310 and 2312. In step 2310, the system updates the data fields of the linked information (e.g., user data, modeling and simulation parameters, simulation output). In step 2312, the user manually updates the digital thread or magic document with text details, referencing data from the DE system. At this point, the process may proceed to step 2332, in which the digital thread and magic document are completed and ready for submission review.

[0317] The AI-assisted digital thread and magic document update process 2334 includes steps 2316, 2318, 2320, 2322, 2324, 2326, 2328, and 2330. In step 2316, the system updates the data fields of the linked information (e.g., user data, modeling and simulation parameters, simulation output). In step 2320, the system suggests relevant data fields in the DE system for the user to include. In step 2324, the user data prompt, digital thread section, and document fields are input to the generator ML model. Separately, in step 2318, training data is prepared. This training data includes a large number (e.g., hundreds or thousands) of exemplary digital threads and associated magic documents related to the data fields of interest. These fields may have been previously tagged, and therefore the data may already be labeled for training. In step 2322, the training data is used to train the generator ML model. Given training data, conventional neural network architectures can become highly sensitive to certain types of documents (e.g., companion magic dogs) and recognize which combinations are meaningful.

[0318] Next, in step 2326, the outputs of steps 2322 and 2324 are input to a generator ML model with NLP / LLM support to assist in the generation of digital thread and magic document text. In step 2328, the system recommends adding text to the digital thread and magic document. In step 2330, the user selects or rejects the recommended text. The system can then fine-tune the generator ML engine by returning to step 2326.

[0319] Finally, in step 2332, the digital thread and magic document are completed, and the submission is ready for review.

[0320] Content and collaborative filtering recommender engine for digital threads In some embodiments of AI-assisted digital thread creation, the machine learning (ML) engine employs both content and collaborative filtering to recommend digital threads. In the user input phase, the user searches for a specific digital thread or compiles various code sections. The system input phase involves labeling a repository of script templates and previous digital threads with metadata, thus defining thread profiles. Data fields from previous threads are checked and removed if they are irrelevant or sensitive.

[0321] In the recommendation phase, the engine filters threads based on profiles, correlates them with user profiles, and assigns a fit metric. The digital thread with the highest fit metric, measured from the user's past actions and profile, is then recommended. The engine not only evaluates and remembers the action steps associated with the current user thread pair, but also updates the fit metric, taking into account the participation of other users engaged in similar tasks.

[0322] These recommended steps involve using a machine learning model trained on a dataset procured from user input, encompassing a library of digital threads, templates, and user profile metadata. As the database expands with more user input and associated threads, specific input-output pairs are selected for fine-tuning. User feedback on digital threads generated by the ML-based generator engine can further refine the system and effectively enhance the digital thread recommendation process.

[0323] Markov Chain Monte Carlo (MCMC) Recommender Engine for Digital Threads In other embodiments, the recommender engine may utilize a Markov Chain Monte Carlo (MCMC) approach to recommend digital threads, using probabilistic processing that depends on the occurrence and sensitivity of specific steps within the digital threads. The process begins with a user input phase, in which the user defines requirements and constraints for a sequence of digital threads, such as specific code snippets, functions, or library calls.

[0324] Following user input, the MCMC phase begins by constructing a state space containing several digital threads that match user-defined requirements. The process then assigns a relevance score and acceptance criteria to each thread, with higher scores indicating a higher probability of selection. Next, a proposal distribution is determined, and threads are randomly selected from the library based on a calculated probability distribution, which may be proportional to the relevance score of each digital thread.

[0325] Next, the MCMC algorithm initializes a Markov chain by selectively choosing an initial state from the state space. The number of iterations of the Markov chain begins, with each iteration randomly drawing a new state from the proposal distribution. Each randomly selected digital thread is evaluated for selection probability and accepted or rejected based on specified acceptance criteria. The algorithm continuously cycles through these iterations until the probabilities converge. The final state is selected from the highest relevance score, representing the step sequence best suited to the digital threads that satisfy user-defined criteria. Finally, the algorithm's final output is the selected digital thread sequence.

[0326] Generative AI approaches for creating digital threads and related magic documents Figure 24 shows a detailed process flow for generating digital threads and associated magic documents (i.e., a generator engine based on generative AI) using a generative AI-assisted approach, as disclosed herein. For example, a user may upload a requirements model and a CAD model design for the engine and then request a summary digital thread and magic document report detailing whether the CAD model parameters meet the specific requirements of the uploaded DE model. The generative engine described in Figure 24 may use the input DE model data to prepare the document, as described below.

[0327] Referring to Figure 24, the process of creating and updating digital threads and associated magic documents begins in the first user input phase 2404, and in step 2406, the user provides text input communicating their purpose for use in the interconnected digital engineering and authentication ecosystem.

[0328] In step 2408, the user may select a specific purpose for the digital thread and magic document (as part of overall authentication or validation). Alternatively, in step 2410, the user may input a specific product design, update, or requirement constraint (e.g., "single engine," "fixed wing," "2,000 pounds," etc.). In step 2412, the user may select a pre-trained LLM (e.g., GPT-3 DaVince model) for baseline document generation / update. In another embodiment, the system may select an appropriate LLM based on the user's profile (e.g., user document profile data).

[0329] In step 2416, baseline metadata is collected from the user, including an ontology of requirements with relevant hierarchies, for authentication or other documentation purposes. For example, such baseline metadata may include requirements defined for Major Capability Acquisition (MCA) or airworthiness requirements (516c airworthiness requirements).

[0330] In the AI-assisted phase 2418 of the process in which a document is created and / or updated, in step 2420, the training set is prepared manually. In step 2420, synthetic data is added to the training set. LLM is suitable for performing this step, and the training data may include examples of previous documents and digital threads, and for this step, metadata may be further added using an external expert user. Alternatively, the synthetic data may be generated using random perturbations on already accepted data elements (e.g., data fields of a document) and then introduced into the training data by a subject expert.

[0331] Next, an LLM fine-tuning step 2426 is performed, which includes fine-tuning the LLM using prompt-response pairs 2428 (for example, from the example database), leading to the development of a custom fine-tuned LLM for the thread creation and documentation process 2430.

[0332] An example of a prompt-response pair is provided below. Prompt: I want to build a fixed-wing aircraft weighing 2000 pounds using a gas turbine engine. I need to demonstrate the engine's safety. Response: Chapter 7 of 516c concerning the safety of propulsion systems states the following requirements: JSSG-2007:A.3.1, A.4.1, A.3.2, A.4.2, A.3.2.1, A.4.2.1, A.3.3.1, A.4.3.1, A.3.3.2, A.4.3.2, A3.4, A.4.4, A.3.5.1, A.4.5.1, A.3.7, A.4.7, A.3.7.2.1, A.4.7.2.1, A.3.11, A.4.11, A.3.12, A.4.12, Table XLIXa USAF PCoE BP 99-06D 14 CFR33.5, 33.35, 33.7, 33.75, 33.8 FAA AC 33-2

[0333] In step 2432, the custom fine-tuned LLM is used to recommend code comments, script code (e.g., code blocks), or text additions (e.g., text blocks) to the digital thread and magic document. In step 2434, Figure 24 shows the use of reinforcement learning from human feedback (RLHF) to loop back to step 2432, thereby showing the user selecting or rejecting the recommended code comments, orchestration code, or text. LLM fine-tuning using a fine-tuning dataset may also be used, as described below. The system loops back to step 2432 as long as new script and / or document additions are requested or the fine-tuning has not reached a pre-specified level of accuracy.

[0334] Note that the LLM in step 2432 utilizes user input as context to distinguish between the update process and the creation process. Updating a digital thread is different from generating a new digital thread in that it checks data fields or orchestration script code that may be required for generation or modification. The user may prompt the ML model (e.g., LLM) to modify a specific orchestration script or data field or magic document portion within a particular document. Alternatively, the ML model (e.g., LLM) may identify data fields, orchestration script code, and magic document portions that are modified after being trained on a dataset of sample unmodified and modified digital thread and magic document pairs. After passing through the updated “requirements,” if the IDEP detects anomalies within the range of values ​​for authentication, the digital thread and magic document portions to be updated may be determined and highlighted by the ML model (e.g., LLM) using user input.

[0335] As an example, an architecture of such fine-tuned LLMs used to generate digital threads and magic documents is described. In one embodiment, the IDEP maintains fine-tuned datasets from various fine-tuned LLMs, each targeting a different domain-specific application. The fine-tuning process includes generating tags for these datasets using the LLM backend, storing these tags in a tag database along with updated metadata, and extracting this information through a cross-platform frontend to create new or updated fine-tuned datasets. These datasets are then reviewed by subject experts before being stored in the database according to CRUD practices. This architecture can utilize commercial off-the-shelf (COTS) infrastructure, such as Azure Database for MongoDB Server, to store the fine-tuning and tag databases. The frontend can integrate datasets and tags using tools such as PyQt, facilitating the creation or updating of fine-tuned datasets.

[0336] Based on the use of platform data, this architecture can be reused with individual fine-tuned datasets to train and create fine-tuned LLM libraries, each customized for specific AI-assisted use cases (e.g., various types of documentation, model sharing) or targeting different DE software or tools.

[0337] In various embodiments, during LLM training and / or fine-tuning, 1. Training data may include previously generated and modified digital threads or documents, as well as creation / modification history of threads or documents for various user profiles. 2. The creation of synthetic data may follow a rule-based approach for permutations of existing data, using an abstract syntax tree for transformations, and a compiler to verify success. 3. The number of prompt-response pairs for fine-tuning the LLM can be increased through permutations following the abstract syntax tree. 4. The system architecture can be reused to train and create finely tuned LLM libraries, each customized for specific AI-assisted use cases (e.g., DE task type, document type, model sharing).

[0338] Furthermore, examples of training data may include any of the following: 1. Tool-specific documentation such as API reference guides, user guides, and tutorials, as well as platform documentation. 2. Technical articles and blog posts that specifically describe digital engineering operations. 3. Online forums (e.g., Stack Overflow) and other Q&A threads. a. The training dataset includes Stack Overflow and other Q&A threads discussing digital engineering documentation. 4. Publicly available documentation, such as descriptions of DE tools and APIs, and other information that can be gathered from publicly available sources.

[0339] In some embodiments, the generation of synthetic data may depend on the following: 1. An abstract syntax tree customized for a specific digital engineering application. 2. Selectively performing permutations on the training data. 3. It is recommended to test the compilation and then add it to the synthesized data. 4. Expert feedback.

[0340] In some embodiments, summaries of digital threads and associated documentation may be generated iteratively, incrementally, or incrementally using a generative AI-based algorithm (e.g., LLM). Specifically, digital threads and associated human-readable documentation on digital threads may be generated from one or more machine-readable DE models by linking between models and model-versus-document, with the assistance of an LLM-based AI module. For example, computer-aided design (CAD) files for airline seat designs and SysML files for requirements may be spliced ​​and linked, and report templates may be spliced ​​and linked, respectively, to generate airline seat certification reports. Once summaries of digital threads and associated documentation are created, they may be automatically and dynamically updated based on revisions to the DE models.

[0341] In an exemplary process, the following steps may be taken: ● LLM Training / Fine-tuning: AI models such as Orchestration Scripts and System Reference Documents (SRDs) LLM (or LLM-OSSRD) may be trained on a limited number of general-purpose LLMs such as GPT4, LLAMA2, and / or MISTRAL, and may be fine-tuned on examples of digital thread orchestration scripts using the relevant SRDs. ● Model splicing: Any input DE model (e.g., CAD model, SysML model) can be spliced, and the resulting API endpoint can be accessed via API calls to product functions (e.g., extracting the center of gravity and weight from a CAD model, exporting requirement parameters from a SysML model). ● Outline generation via LLM: API responses may be added to prompts to link the DE model to a digital thread and to generate the associated documentation summary. ● Generation of Digital Threads and Documentation via LLM by Section: The LLM-OSSRD, fine-tuned above, may prompt for one part of a digital thread and the corresponding section of the documentation summary, either part-by-part or section-by-section, until all parts of the digital thread and associated documentation are generated. As previously mentioned, the digital thread may be represented by an orchestration script, which may constrain multiple sections, such as code blocks with transaction / execution metadata and text blocks (see Figure 27). When executed, the code blocks may perform specific subtasks within the orchestration script. The code blocks may contain actual code or links to code. The text blocks provide context, parametric, requirements-related, and / or authentication-related information about the linked DE model. Each block or group of consecutive blocks may be generated via LLM at the appropriate prompt. Similarly, parts of the associated documentation summary may be generated iteratively. In some embodiments, the digital thread section (e.g., a code block) and the corresponding related documentation summary section (e.g., a description of the code block's function and input / output range) may be generated by the LLM as a single output. The motivation behind this iterative approach is that LLMs typically have token limitations on input sequences, and prompt generation must take this limitation into account, but only aggregate a subset of DE model data related to a single part. ● Digital threading and documentation compilation: All parts are compiled into a complete draft.

[0342] In some embodiments, the associated documentation summary or companion magic document ("magic doc") provides auditability and traceability by logging the execution of portions of the orchestration script. These features allow users or third parties to record and audit the progress of the digital thread and interactions with other entities and files, thereby enabling accountability and trust. To implement these features, the magic doc includes an API endpoint to a human-readable text block, and when a portion of the orchestration script (e.g., a code block) is executed, the associated audit log may be added to the magic doc using that API endpoint. This addition may be persistent and / or impossible to erase by third parties. This audit log may include endpoint metadata associated with the digital thread transaction log and / or the call to the API endpoint. Exemplary endpoint metadata may include the model owner organization, model owner ID, user ID, user access rights, device ID, device location according to IP number and geographic location identifier, as well as the IDs of the model splice and splice function, transaction commands associated with the model splice and splice function calls, the time associated with each transaction command, and values ​​associated with the transaction. Other examples may include the function ID, the type of method of calling, the transaction start time, the transaction end time, the duration, the parameters of the call made by the model splice and splice function, the success of the call (e.g., "true", "false", or "NULL"), the CPU cost time in monetary terms, time, and / or cycles, and the GPU cost time in monetary terms, time, and / or cycles. Other examples are also possible.

[0343] Domain-Specific Language (DSL) Generator Engine for Digital Threads In certain embodiments, IDEP incorporates a generator engine for digital threads using a domain-specific language (DSL). Digital threads naturally fit within the rule-based generation system of the DSL.

[0344] In an exemplary embodiment, the user may specify requirements for the orchestration script and select an appropriate DSL parser. The user may then develop DSL rules that describe the orchestration script, detailing how various software components or services interact. The user may also refer to the necessary code documentation for API scripts of various DE tools.

[0345] During the operational phase, the DSL engine may adopt these rules to manage the workflow. The DSL engine may scan a database of scripts or service definitions to identify matches that meet user-specified criteria. These elements can then be compiled to generate a functional digital thread, which provides the DSL with strict control over the orchestration process.

[0346] Generally, DSL-based generator engines are memoryless and may not benefit from the context available to ML-based or transformer-based generator engines. However, a set of rules for "types" of digital thread templates for a certain range of DE tasks may be provided in a digital thread template database. Based on the digital thread template data fields to be filled, rules may be provided for traversing a data field tree, building the tree, and then assigning values ​​or labels as it is traversed. This employs a dynamic tree approach, in which case the rules depend on the needs of the digital threads being generated.

[0347] Exemplary Digital Thread Graphical User Interface Exemplary Digital Engineering Verification and Certification Process Figure 25 shows a graphical user interface (GUI) associated with an exemplary digital thread for verifying and authenticating requirements within IDEP, as disclosed herein. Figure 25 also shows an example of an unmanned aerial vehicle (e.g., a drone) undergoing digital authentication.

[0348] A series of exemplary displays 2500 shown on a user device illustrate the authenticated digital thread, as illustrated by the examples disclosed herein. Note that in embodiments including an artificial user interfaced with a computing system via API 206B, the artificial user can directly process the digital computer files received via API 206B without further visualization, and therefore no display is necessary. The series of exemplary displays 2500 may correspond to the exemplary digital thread workflow described in relation to Figures 15–20. Again, these displays are not intended to be limiting, but merely illustrate the types of user experiences that a user 204 (and especially a human user) may encounter while implementing the digital thread for the interconnected digital engineering and authentication ecosystem 200. As described herein, the series of exemplary displays 2500 highlight the ease of use of the ecosystem 200 and the avoidance of the complexity that would require a user to interface with individual digital engineering tools and manually review complex common V&V products to evaluate whether a product prototype should be authenticated.

[0349] Display 2502 shows a login screen that may be displayed on user device 206A. The login screen may prompt user 204 to enter user credentials (e.g., username and password) to access computing system 208 and the rest of interconnected digital engineering and authentication ecosystem 200. User credentials associated with user 204 may perform various functions. For example, as mentioned above, user credentials may be associated with the user's skill level, which may control which functions of ecosystem 200 user 204 can access. In some embodiments, user credentials may additionally or alternatively be associated with the user's affiliated entities (e.g., specific companies and / or organizational entities). This may manage previously designed products 208 and / or solutions that the user may search for and / or that may be proposed by computing system 208. In general, user credentials may help ensure that user 204 can access only information within ecosystem 200 that user 204 is entitled to and / or authorized to access.

[0350] After user 204 logs in from user device 206A, user device 206A may be used to develop a digital prototype of a product. For example, display 2504 shows a modeling screen that user 204 can see while developing a digital model of a UAV (e.g., using a CAD tool). Once the prototype is developed, the user may upload prototype data, such as CAD files and / or MBSE files, to computing system 208. Therefore, display 2506 shows a screen that may prompt user 204 to upload MBSE files and CAD files to computing system 208.

[0351] Once the user uploads MBSE and CAD files to the computing system 208, the computing system 208 may perform several steps to evaluate the prototype against one or more requirements identified in the Common V&V Product and generate a report summarizing the evaluation. In the exemplary embodiment of Figure 25, the evaluation is performed with regard to weight and center of gravity certification. In doing so, the computing system 208 may also communicate with the Digital Engineering Tools 202 and the Common V&V Product 210 repository, which themselves may perform actions to facilitate the evaluation of the prototype. These steps may take some time to complete (e.g., from a few seconds to several hours), during which time the display...

Claims

1. A non-temporary physical storage medium for storing program code, wherein the program code is executable by the hardware processor to cause the hardware processor to execute a computer execution process for generating software code definition digital threads, and the program code is Training a script-generating machine learning (ML) model using a training dataset comprising a set of training triplets each containing a sample intent input, a corresponding sample model representation set, and a corresponding sample platform orchestration script, wherein the sample platform orchestration script connects the models in the corresponding sample model representation set to perform the corresponding sample intent input. Receiving a first model representation of the first engineering model, Receiving the second model representation of the second engineering model, Receiving intent input, Using the script generation ML model, a platform orchestration script is generated that connects the first model representation and the second model representation based on the intent input, wherein the platform orchestration script performs the intent input, and the generation is performed. The platform orchestration script is stored as the software code definition digital thread, The non-temporary physical storage medium includes code for performing the following actions.

2. Receiving feedback data regarding the aforementioned platform orchestration script, The script generation ML model is trained / fine-tuned based on the aforementioned feedback data. A non-temporary physical storage medium according to claim 1, further comprising program code for performing the following.

3. To provide a user interface coding environment in the Interconnected Digital Engineering Platform (IDEP), Receiving multiple user selections of the first engineering model and the second engineering model, wherein the first engineering model and the second engineering model are selected by the user, Receiving multiple corresponding model representations from the first engineering model and the second engineering model, Receiving user-defined code for user-defined platform orchestration scripts, Determining and / or receiving the corresponding intent input, The user-defined platform orchestration script determines the corresponding model representation endpoint used in the user-defined code, To generate the training dataset, the first engineering model and the second engineering model, the first model representation and the second model representation, the corresponding intent input, the corresponding model representation endpoint, and the user-defined platform orchestration script are recorded. The training dataset for training the script generation ML model is stored, A non-temporary physical storage medium according to claim 1, further comprising program code for performing the following.

4. The non-temporary physical storage medium according to claim 1, wherein connecting the first model representation and the second model representation based on the intent input includes linking a first endpoint of the first model representation and a second endpoint of the second model representation based on the intent input.

5. The non-temporary physical storage medium according to claim 4, further comprising program code for evaluating the sufficiency of the first and second engineering models in an interconnected digital engineering platform (IDEP) for performing the intent input using an adequate machine learning (ML) model.

6. The non-temporary physical storage medium according to claim 5, further comprising program code for determining a first endpoint in the first model representation relating to the intent input, using a recommender ML model or the script generation ML model in response to sufficiency being determined.

7. The non-temporary physical storage medium according to claim 6, further comprising program code for determining the relationship between the first endpoint and the second endpoint based on the intent input using the recommender ML model.

8. The non-temporary physical storage medium according to claim 1, wherein the platform orchestration script includes script code for reading data from the first model representation and / or the second model representation.

9. The non-temporary physical storage medium according to claim 1, wherein the platform orchestration script includes script code for writing data to the first model representation and / or the second model representation.

10. The non-temporary physical storage medium according to claim 1, wherein the platform orchestration script includes an input for a second model representation connected to the output of the first model representation.

11. A non-temporary physical storage medium according to claim 10, further comprising program code for performing the execution of the platform orchestration script for the second model representation, wherein the output from the first model representation is the input for the second model representation.

12. The non-temporary physical storage medium according to claim 1, wherein the platform orchestration script includes script code for reading data from the first model representation, performing calculations on the data, and writing the results of the calculations to the first model representation and / or the second model representation.

13. The code further includes program code for receiving a third model representation of the third engineering model, The non-temporary physical storage medium according to claim 1, wherein the platform orchestration script further links the first model representation and / or the second model representation with the third model representation.

14. The non-temporary physical storage medium according to claim 1, further comprising program code for executing the platform orchestration script by calling one or more API endpoints or SDK endpoints associated with the first model representation and / or the second model representation.

15. The non-temporary physical storage medium according to claim 1, further comprising program code for determining a recommended third engineering model based on the first engineering model, the second engineering model, and the training dataset, using an AI algorithm.

16. The non-temporary physical storage medium according to claim 1, wherein the first engineering model and / or the second engineering model is a human-readable document file.

17. Receiving document templates and, Analyzing the document template using the Interconnected Digital Engineering Platform (IDEP), Using an AI model, determine the output data from the first model representation and / or the second model representation necessary to generate the document file, In order to generate the output required for the document file, appropriate actions on the first model representation and / or the second model representation are performed using a predetermined sequence based on the requirements of the document template, The document file is generated by assembling the output from the document template and the first model representation and / or the second model representation. A non-temporary physical storage medium according to claim 16, further comprising program code for performing the following.

18. The non-temporary physical storage medium according to claim 1, further comprising program code for predicting changes in one or more items selected from the group consisting of the first model representation of the first engineering model and the second model representation of the second engineering model, based on a change in one of the items selected from the group consisting of the first model representation of the first engineering model and the second model representation of the second engineering model.

19. The non-temporary physical storage medium according to claim 1, further comprising program code for predicting changes in the first model representation of the first engineering model based on changes in the second model representation of the second engineering model.

20. A non-temporary physical storage medium according to claim 1, further comprising program code for calling a second software code definition digital thread.

21. The non-temporary physical storage medium according to claim 1, wherein one of the first engineering model and / or the second engineering model includes a neural network model.

22. A non-temporary physical storage medium according to claim 1, comprising using an AI model to generate a magic document associated with the software code definition digital thread, wherein the magic document includes an API endpoint to a human-readable text block, and the magic document further includes program code for doing so, which is updated in an audit log in response to the execution of at least a portion of the platform orchestration script using the API endpoint.

23. The non-temporary physical storage medium according to claim 1, wherein the platform orchestration script includes a code block, the code block is associated with an information security tag, and the information security tag indicates a restriction on the execution of the code block.

24. The first model representation is the first model splice, Receiving a first engineering model file of the first engineering model having a DE model type, wherein the first engineering model file is in a native file format, Extracting model data from the first engineering model file in the aforementioned native file format, The aforementioned model data is stored in the model data storage area, To generate one or more externally accessible splice functions that enable external access to one or more digital artifacts derived from the model data stored in the model data storage area, wherein the one or more externally accessible splice functions provide addressable application programming interface (API) endpoints or software development kit (SDK) endpoints that are accessible to third-party applications and users, and the API endpoints or SDK endpoints enable access to the digital artifacts without accessing the entire first engineering model file and without requiring direct involvement from the third-party applications and users using DE tools associated with the DE model type, Generating a first model splice of the first engineering model, wherein the first model splice includes access to selected portions of the one or more digital artifacts, the first model splice includes access to at least one of the one or more externally accessible splice functions, the first model splice is accessible by the third-party application and the user via the API endpoint or the SDK endpoint, and the API endpoint or the SDK endpoint provides a unified programming interface to a shareable model splice generated from a DE model having the DE model type, A non-temporary physical storage medium according to claim 1, further comprising program code for generating the first model splice of the first engineering model using program code for performing the above.

25. A computer implementation method for generating a software code definition digital thread, Training a script-generating machine learning (ML) model using a training dataset comprising a set of training triplets each containing a sample intent input, a corresponding sample model representation set, and a corresponding sample platform orchestration script, wherein the sample platform orchestration script connects the corresponding sample model representation set to perform the corresponding sample intent input. Receiving a first model representation of the first engineering model, Receiving the second model representation of the second engineering model, Receiving intent input, Using the script generation ML model, a platform orchestration script is generated that connects the first model representation and the second model representation based on the intent input, wherein the platform orchestration script performs the intent input, and the generation is performed. The platform orchestration script is stored as the software code definition digital thread, The computer implementation method, including the above.

Images