Identifier-based tracking and verification of machine learning model modifications

Abstract

The disclosure relates to identifier-based tracking and verification of machine learning model modifications, and in various examples, tracking and verifying modifications of a machine learning model using a unique identifier are described herein. For example, the systems and methods described herein may generate unique identifiers, such as checksums, hashes, etc., that track modifications to the model. For example, if a first version of the model is modified to generate a second version of the model, such as by performing fine tuning, optimization, quantization, and / or any other modification technique, data representing the modification may be obtained. The data may then be used to generate information describing the modification. Additionally, information may be encoded using one or more encoders and / or processed using one or more algorithms to generate a unique identifier for the second version of the model. The present disclosure also describes systems and methods for performing various tasks using unique identifiers.

Description

Background Technology

[0001] Online repositories allow users to download, modify (e.g., update, optimize, etc.), and share models (e.g., machine learning models, neural networks, classifiers, etc.). For example, a user can use an online repository (such as by performing fine-tuning, optimization, and / or quantization) to modify a first version of a model to generate a second version. The user can then share this second version of the model with additional users using the online repository. However, traditional systems that provide online repositories for models may not offer any and / or sufficient mechanisms to track and / or verify modifications performed on the model. For example, if a user uses fine-tuning to generate a new version of the model, other users may not be able to identify the dataset used to perform the fine-tuning, the accuracy of the model after the fine-tuning, and / or the new version of the model generated by performing the fine-tuning. This can make it difficult for traditional systems to track the integrity and / or origin of the model.

[0002] To compensate for these shortcomings, some traditional systems may store model cards that include information about available models. For example, model cards might contain various information about a specific model (such as the model's development process, training data, performance metrics, potential biases, limitations, intended use cases, and / or out-of-scope applications), which could allow end users to make informed decisions about model deployment and / or use. However, when the model is modified to generate a new version, these model cards cannot be updated with information describing the modification. Additionally, even when the model cards are updated, traditional systems require users to manually identify the information associated with the modification and input that information into the model card. Thus, traditional systems are prone to errors, such as users inputting incorrect information into the model cards and / or failing to identify information describing the modification. Summary of the Invention

[0003] The embodiments of this disclosure relate to identifier-based tracking and verification of machine learning model modifications. For example, the systems and methods described herein can generate unique identifiers for tracking model modifications, such as checksums, hashes, etc. For instance, if a first version of a model is modified (e.g., updated, changed, optimized, enhanced, etc.) to generate a second version of the model (e.g., by performing fine-tuning, optimization, quantization, and / or any other modification techniques), data representing the modification can be obtained. This data can then be used to generate information describing the modification (e.g., text, messages, comments, reviews, etc.). Additionally, the information can be encoded using one or more encoders and / or processed using one or more algorithms (e.g., one or more hash algorithms) to generate a unique identifier for the second version of the model. This disclosure then describes systems and methods for performing various tasks using unique identifiers, such as verifying the integrity of different versions of the model, providing information about modifications, etc.

[0004] Compared to conventional systems, in some embodiments, the system of this disclosure generates a unique identifier that can be used to track modifications made to a model, verify the integrity of the model, and / or identify a specific version of the model for further modification and / or use. For example, if a user is attempting to obtain a model from an online repository, the user can use the unique identifier to identify information such as how the model version was modified, the performance score of the model version, the generation date of the model version, the identifier of the generator of the model version, and / or any other modification information. Thus, the user can use the unique identifier to select a version of the model to retrieve and / or verify the integrity of the model after retrieval. In other words, the system of this disclosure can use the unique identifier to guarantee the integrity and / or origin of models provided by an online repository.

[0005] The embodiments of this disclosure also relate to the dynamic updating of machine learning model cards based on model changes. For example, the systems and methods described herein can automatically update model cards using supplementary information about modifications that have occurred to the model. To update the model card, a unique identifier (such as by performing fine-tuning, optimization, quantization, and / or any other modification techniques) can be used to identify modifications that have occurred to the model in order to generate a new version of the model. Supplementary information about the modifications is then obtained (such as by decoding the unique identifier, testing the new version of the model, and / or using data describing how the model was modified). The model card associated with the model can then be automatically updated using the supplementary information. Additionally, this process can be repeated as additional modifications to the model occur to generate a new version of the model.

[0006] Compared to conventional systems, in some embodiments, the system of this disclosure automatically determines modifications to the model, obtains supplementary information associated with the modifications, and then uses the supplementary information to update the model card. For example, and as described above, conventional systems may store model cards that include information about the initial version of the model. However, when the model is modified, the model cards stored in conventional systems are typically not updated and / or require user input for updating, which is prone to errors. Thus, the system of this disclosure provides an improvement by continuously updating the model card to include supplementary information about the new version of the model. Additionally, by using the techniques described herein to obtain supplementary information, the system of this disclosure ensures that the model card is accurately updated to reflect information about the actual characteristics and / or performance of the model. Attached Figure Description

[0007] The following describes in detail, with reference to the accompanying drawings, the system and method for tracking and validating modifications to machine learning models, wherein:

[0008] Figure 1Examples of processes for tracking and verifying modifications to a model using identifiers, according to some embodiments of the present disclosure, are illustrated.

[0009] Figure 2 Examples of different modified versions of the storage model according to some embodiments of the present disclosure are illustrated;

[0010] Figure 3 Examples of obtaining data representing characteristics and / or modifications associated with a model, according to some embodiments of this disclosure;

[0011] Figure 4 Examples of generating structured information associated with a version of a model according to some embodiments of this disclosure are illustrated;

[0012] Figure 5 The illustration shows examples of generating unique identifiers associated with various versions of a model according to some embodiments of this disclosure;

[0013] Figure 6 The illustration shows examples of providing structured information associated with various versions of a model according to some embodiments of this disclosure;

[0014] Figure 7 The illustration shows an example of a process for automatically updating a model card to include supplementary information on modifications to the model, according to some embodiments of the present disclosure.

[0015] Figure 8 Examples of decoding a unique identifier to obtain supplementary information associated with modifications to the model are illustrated according to some embodiments of this disclosure;

[0016] Figure 9 The illustration shows examples of test models according to some embodiments of the present disclosure for determining performance information;

[0017] Figures 10A-10C Examples of updating model cards to include supplementary information associated with modifications to the model, according to some embodiments of this disclosure, are illustrated.

[0018] Figure 11 The illustration shows a flowchart of a method for generating a unique identifier associated with modifications to a model, according to some embodiments of the present disclosure;

[0019] Figure 12 The illustration shows a flowchart of a method for using a unique identifier to provide information associated with a model, according to some embodiments of the present disclosure;

[0020] Figure 13 The illustration shows a flowchart of a method for verifying a model using a unique identifier, according to some embodiments of the present disclosure;

[0021] Figure 14-15 The illustration shows a flowchart of a method for updating a model card with information describing modifications to the model, according to some embodiments of the present disclosure;

[0022] Figure 16 This is a block diagram of an example computing device applicable to implementing at least some embodiments of the present disclosure; and

[0023] Figure 17 This is a block diagram of an example data center applicable to implementing at least some embodiments of the present disclosure. Detailed Implementation

[0024] This disclosure discloses systems and methods related to tracking and validating modifications to machine learning models. For example, one or more systems may provide models (e.g., machine learning models, neural networks, classifiers, etc.) using one or more techniques (such as managing one or more repositories, libraries, archives, databases, etc.). For example, one or more systems may provide a model repository that allows users to at least store, share, manage, access, modify, and / or use models. As described herein, models may be modified (e.g., updated, changed, optimized, enhanced, etc.) using one or more techniques. For example, models may be modified at least by: fine-tuning the model to perform a specific task, optimizing the model to perform using specific hardware components (e.g., a specific type of processor, a specific type of memory, etc.), hyperparameter tuning the model, performing quantization on the model, and / or performing any other type of model modification technique.

[0025] For a specific example, one or more systems may use a model repository to receive and store a first version of a model. Then, one or more first users may use one or more first computing devices to access the first version of the model in the model repository, modify the first version of the model using one or more techniques to generate a second version of the model, and then upload the second version of the model back to the model repository. Then, one or more second users may use one or more second computing devices to access the first or second version of the model in the model repository, modify the first or second version of the model using one or more techniques to generate a third version of the model, and then upload the third version of the model back to the model repository. These processes may then continue to be repeated, such that the model repository stores versions of the model that have been modified by different users using various techniques. Therefore, for one or more systems, it may be important to provide at least a mechanism for tracking and / or verifying modifications that occur to the model stored in the model repository.

[0026] Thus, when a model is modified to a new version, one or more systems can generate, receive, obtain, and / or retrieve data representing characteristics associated with the model, modifications made to the model, information about the user who performed the modifications, and / or any other information associated with the new version of the model. For example, in some examples, the data may represent the architecture associated with the model, one or more datasets used to train the model, one or more risk scores associated with the model, one or more licenses associated with the model, one or more performance scores associated with the model, the hardware on which the model is optimized to run, one or more changes made to the model, the identity of the user who performed the modifications, and / or the date and / or time of the modifications. In some examples, one or more systems may automatically generate, receive, obtain, and / or retrieve data during and / or after the model is modified. In some examples, one or more systems may generate, receive, obtain, and / or retrieve data based on one or more events, such as a new version of the model being uploaded.

[0027] Then, one or more systems may process at least a portion of the data to generate structured information describing the characteristics and / or modifications associated with the new version of the model. For example, in some examples, one or more systems may generate information (e.g., messages) summarizing changes made (e.g., fine-tuning on a new dataset, quantization using a specific type of processor for inference, etc.), information describing details of the modifications to the model (e.g., changes to architecture, risk scores, licenses, etc.), information describing performance improvements associated with the model (e.g., improved accuracy, improved accuracy with specific hardware, etc.), information describing the date the modifications occurred, information describing one or more users who performed the modifications, and / or information describing any other characteristics and / or modifications associated with the model.

[0028] Then, one or more systems can generate unique identifiers that at least track modifications associated with a new version of the model. As described herein, unique identifiers can include, but are not limited to, checksums, hashes, codes, letter identifiers, numeric identifiers, alphanumeric identifiers, and / or any other type of identifier that can be used to track modifications associated with the model. For example, one or more systems can use one or more encoders to encode at least a portion of structured information describing the characteristics and / or modifications associated with a new version of the model. In some examples, one or more encoders can encode the structured information into a compressed format and / or binary, which may also be referred to as encoded information. One or more systems can then combine the encoded information with the model's configuration to generate a unique identifier associated with the model. For example, one or more systems can use one or more hash algorithms (e.g., SHA-256, Blake3, etc.) to generate checksums based on the combined encoded information and the model's configuration. By performing such a process, a unique identifier can at least represent information describing the characteristics and / or modifications associated with a new version of the model.

[0029] Then, one or more systems can use one or more technologies to store unique identifiers and / or data associated with the model. For example, in some examples, one or more systems can store unique identifiers and / or data along with the model (such as using a model repository). Additionally or alternatively, in some examples, one or more systems can send unique identifiers, data, and / or the model to one or more external platforms that are also used to store and / or provide the model. In other words, one or more systems can synchronize modifications made to the model with other external platforms.

[0030] As described herein, one or more systems and / or one or more other computing devices can then perform one or more tasks using unique identifiers. For example, one or more systems and / or one or more computing devices can use a unique identifier associated with a model to at least verify the integrity of the model. As described herein, in some examples, one or more systems and / or one or more computing devices can verify the integrity of the model by comparing a newly generated unique identifier with a unique identifier associated with the model. For example, one or more systems can (e.g., in response to a receiving request) provide a model, data, and unique identifiers to a computing device. One or more computing devices can then generate new unique identifiers associated with the model using one or more techniques described herein. Additionally, when these unique identifiers match, one or more computing devices can verify the integrity of the model; and / or when these unique identifiers do not match, the integrity of the model cannot be verified. This is because in some examples, the unique identifiers cannot match when the model has been further modified (e.g., tampered with, such as by performing unauthorized modifications).

[0031] Additionally, one or more systems may use unique identifiers to provide information associated with an available model (e.g., a model stored in a model repository). For the first example, one or more systems may use a unique identifier associated with a version of the model to retrieve information describing the characteristics and / or modifications associated with the model. In some examples, such as when the unique identifier includes a checksum, one or more systems may retrieve information by decoding the checksum using one or more decoders. The information may then be provided to one or more users. In this way, one or more users can identify important information associated with the version of the model, such as modifications made to the model, the characteristics of the modified model, the impact of the modifications on the model's performance, the modifier, the modification time, etc.

[0032] For the second example, one or more systems can use unique identifiers associated with multiple versions of a model to retrieve information describing features and / or modifications associated with one or more versions (e.g., each version). One or more systems can then provide this information to one or more users, where it provides a clear audit trail of the model's history. For example, one or more users can use this information to identify changes that occurred at each modification step associated with the model. Additionally, one or more users can use this information to identify the version of the model to retrieve for performing one or more tasks. For example, if the latest version of the model includes modifications that need to be reverted, one or more users can use this information to identify which version introduced the modifications and / or retrieve previous versions of the model that did not include those modifications.

[0033] Additionally, in some examples, one or more systems may use other techniques to track modifications made to the model. For example, one or more systems may generate and / or update model cards associated with the model. As described herein, model cards may initially include fields that provide initial information associated with the model, such as attributes, model architecture, intended use cases, extended applications, inputs, outputs, intended users, model performance for different groups, training, constraints, computational requirements, etc. Additionally, attributes may include, but are not limited to, the model's name and / or identifier, one or more names and / or identifiers of one or more datasets used to train the model, one or more sizes of one or more datasets, the number of epochs used for training, the license type associated with the model, one or more risk scores associated with the model, one or more bias scores associated with the model, one or more losses associated with the model, and / or any other type of attribute.

[0034] Then, one or more systems can update the model card to include information associated with modifications that have occurred regarding the model. For example, one or more systems can modify the model card to include additional fields representing information related to the modification, such as the type of modification, a unique identifier associated with the model, the date of the modification, performance information associated with the modification (e.g., accuracy score, precision score, recall score, etc.), the date the modified model was deployed, the hardware on which the model was optimized, performance characteristics associated with the model executed on the hardware (e.g., throughput, latency, power consumption, memory utilization, etc.), and / or any other information associated with the modification.

[0035] For example, to update a model card, one or more systems can first determine that the model is being updated and / or has already been updated to generate a new version of the model. In some examples, one or more systems can determine that the model has been updated based on a unique identifier associated with the new version of the model. For example, one or more systems can determine that the unique identifier is new, such that the unique identifier indicates that a modification has occurred. In some examples, one or more systems can determine that the model has been updated based on determining that a user is using a computing device to modify the model and / or has used a computing device to modify the model. However, in some examples, one or more systems can determine that the model has been updated based on the fact that a new version of the model is being uploaded to one or more systems (and / or another external repository). While these are just a few example techniques for how one or more systems can determine that a model has been updated, in other examples, one or more systems may use additional and / or alternative techniques.

[0036] Then, one or more systems can determine supplementary information associated with modifications that have occurred with respect to the model. In some examples, one or more systems can use a unique identifier to determine at least a portion of the supplementary information. For example, one or more systems can decode the unique identifier to obtain structured information associated with a new version of the model. In some examples, one or more systems can use data representing characteristics associated with the model, modifications that have occurred with respect to the model, and / or information about the users who performed the modifications to the model to determine at least a portion of the supplementary information, where the data was originally obtained to generate the unique identifier. However, in some examples, one or more systems can determine at least a portion of the supplementary information based on performing one or more tests associated with the model, such as determining performance information associated with a new version of the model. While these are just a few example techniques for how one or more systems can determine supplementary information associated with modifications to the model, in other examples, one or more systems may use additional and / or alternative techniques.

[0037] Then, one or more systems can update the model cards associated with the model using supplementary information. For example, one or more systems can identify the model cards associated with the model (e.g., by using the model's identifier, the model card's identifier, and / or any other type of identifier). One or more systems can then add at least a portion of the supplementary information to the model cards. Additionally, these processes may continue to repeat as the model is modified to generate additional versions of the model. For example, if the model is optimized for ten different types of processors, one or more systems can modify the model cards to include supplementary information associated with each optimization in the model's optimization. Furthermore, these processes may continue to repeat to update the model cards for any number of models.

[0038] In some examples, one or more systems may generate a single model card and then update it with supplementary information associated with each modification that occurs to the model. However, in other examples, one or more systems may generate multiple model cards associated with modifications that occur to the model. For the first example, one or more systems may generate a first model card including initial information associated with a first version of the model and supplementary information associated with a first modification to generate a second version of the model, generate a second model card including initial information associated with the first version of the model and supplementary information associated with a second modification to generate a third version of the model, and so on. For the second example, one or more systems may generate a first model card including a first modification of a first chain of modifications used to generate a version of the model, a second model card including a second modification of a second chain of modifications used to generate additional versions of the model, and so on.

[0039] As described herein, one or more systems can then use the model card to perform one or more processes. For example, in some examples and similar to unique identifiers, one or more systems may locally store the model card associated with the model and / or send the model card to one or more external platforms that are also used to store and / or provide the model. In other words, one or more systems may synchronize updates to the model card with external platforms. Additionally, in some examples, one or more systems may provide the model card to the user of the model. In this way, the user can identify important information associated with the model, such as modifications made to the model, the characteristics of the modified model (such as on which hardware the model is optimized for implementation), the impact of the modifications on the model's performance, the modifier, and the time of modification.

[0040] In some examples, one or more systems may provide one or more mechanisms for viewing and / or searching model cards. For instance, since model cards can be updated to include information involving numerous modifications (e.g., hundreds of modifications, etc.) associated with the model, one or more systems may generate a user interface that allows users to quickly search model cards to identify specific information. For the first example, such as when the model card includes a text file, the user interface may include one or more interface elements (e.g., buttons, search bars, sliders, etc.) that the user can use to select different versions of the model. The user interface can then provide information related to the selected version of the model. For the second example, one or more systems may convert the text file associated with the model card into a user-searchable format.

[0041] The systems and methods described herein can be used, but are not limited to, by non-autonomous vehicles or machines, semi-autonomous vehicles or machines (e.g., in one or more adaptive driver assistance systems (ADAS)), autonomous vehicles or machines, manned and unmanned robots or robotic platforms, warehouse vehicles, off-road vehicles, vehicles coupled to one or more trailers, airships, boats, shuttles, emergency response vehicles, motorcycles, electric or motorized bicycles, aircraft, engineering vehicles, underwater vehicles, drones, and / or other vehicle types. Furthermore, the systems and methods described herein can be used for a variety of purposes, by way of example but not limited to: machine control, machine motion, machine driving, synthetic data generation, model training, perception, augmented reality, virtual reality, mixed reality, robotics, safety and supervision, simulation and digital twins, autonomous or semi-autonomous machine applications, deep learning, environmental simulation, object or actor simulation and / or digital twins, data center processing, conversational AI, optical transport simulation (e.g., ray tracing, path tracing, etc.), collaborative content creation for 3D assets, cloud computing, and / or any other suitable application.

[0042] The disclosed embodiments can be included in a variety of different systems, such as automotive systems (e.g., control systems for autonomous or semi-autonomous machines, perception systems for autonomous or semi-autonomous machines), systems implemented using robots, aviation systems, medical systems, boating systems, smart area monitoring systems, systems for performing deep learning operations, systems for performing simulation operations, systems for performing digital twin operations, systems implemented using edge devices, systems implementing language models (e.g., small language models (SLMs) (including mini language models and very small language models), large language models (LLMs), systems implementing one or more visual language models (VLMs), etc. Systems that implement one or more multimodal language models, systems that use or deploy one or more inference microservices, systems that contain one or more machine learning models deployed in services or microservices and OS-level virtualization packages (e.g., containers), systems that contain one or more virtual machines (VMs), systems that perform synthetic data generation operations, systems that are at least partially implemented in a data center, systems that perform conversational AI operations, systems that perform optical transmission simulations, systems that perform collaborative content creation for 3D assets, systems that perform generative AI operations, systems that are at least partially implemented using cloud computing resources, and / or other types of systems.

[0043] refer to Figure 1 , Figure 1 Examples of a process 100 for tracking and verifying modifications to a model using identifiers, according to some embodiments of this disclosure, are illustrated. It should be understood that the arrangements and other arrangements described herein are illustrative only. Other arrangements and elements (e.g., machines, interfaces, functions, sequences, functional groups, etc.) may be used in addition to or as alternatives to the arrangements and elements shown, and some elements may be omitted entirely. Furthermore, many of the elements described herein are functional entities that can be implemented as discrete or distributed components, or in combination with other components, and in any suitable combination and location. The various functions described herein as being performed by entities can be performed by hardware, firmware, and / or software. For example, various functions can be performed using a processor that executes instructions stored in memory.

[0044] The process 100 may include: one or more systems 102 storing model 104 in one or more databases 106. As described herein, model 104 may include, but is not limited to, machine learning models, neural networks, classifiers, etc. Additionally, one or more systems 102 may include, or manage, a portion of, and / or manage a repository, library, archive, database, etc. associated with model 104. For example, one or more systems 102 may provide a model repository that allows users to at least store, share, manage, access, modify, and / or use model 104. As described herein, model 104 may be modified using one or more techniques, such as updating, changing, optimizing, and / or enhancing. For example, model 104 can be modified at least by: fine-tuning the model to perform a specific task, optimizing model 104 to implement using specific hardware components (e.g., processor, memory, etc.), performing hyperparameter tuning on model 104, performing quantization on model 104, and / or performing any other type of model modification technique.

[0045] For a specific example, one or more systems 102 may use a model repository to receive and store a first version of model 104. Then, one or more first users may use one or more first computing devices 108 to access the first version of the model in the model repository, modify the first version of model 104 to generate a second version of model 104, and then upload the second version of model 104 back to the model repository. In some examples, one or more first users may include authorized and / or certified users capable of modifying model 104 and / or uploading the second version of model 104 back to the model repository. In some examples, one or more first users may use any technique, such as fine-tuning model 104 to perform new tasks and / or optimizing model 104 to be implemented using specific hardware, to modify the first version of model 104.

[0046] Next, one or more second users can use one or more second computing devices 108 to access a first version and / or a second version of model 104 on the model repository, modify the first and / or second versions of model 104 to generate a third version of model 104, and then upload the third version of model 104 back to the model repository. In some examples, one or more second users can include authorized and / or certified users capable of modifying model 104 and / or uploading the third version of model 104 back to the model repository. In some examples, one or more second users can use any technique (such as fine-tuning model 104 to perform new tasks and / or optimizing model 104 to be implemented using specific hardware) to modify the first and / or second versions of model 104. These processes can then continue to be repeated, such that the model repository stores versions of model 104 that have been modified by different users using various techniques.

[0047] for example, Figure 2 Examples of storing different versions of a modified model according to some embodiments of the present disclosure are illustrated. As shown, the model repository may store a first version 202(1) of the model (e.g., an initial version of the model), a second version 202(2) of the model generated by modifying the first version 202(1) of the model 204(1), a third version 202(3) of the model generated by modifying the second version 202(2) of the model 204(2), and a fourth version 202(4) of the model generated by modifying the third version 202(3) of the model 204(3). Additionally, the model repository may store a fifth version 202(5) of the model generated by modifying the second version 202(2) of the model 204(4) and a sixth version 202(6) of the model generated by modifying the fifth version 202(5) of the model 204(5). Thus, a user can communicate with the model repository using a computing device to access, download, retrieve, receive, modify, and / or use one or more versions of the model 202(1)-(6).

[0048] Return to reference Figure 1For example, since model 104 can be used with different technologies and / or modified by various users, one or more systems 102 can provide one or more mechanisms to track, identify, and / or verify modifications associated with model 104. For instance, the process 100 may include using one or more modification components 110 configured to generate, receive, obtain, and / or retrieve model data 112 representing characteristics associated with model 104, modifications made to model 104, information about the users who performed the modifications to model 104, and / or any other information associated with model 104. For example, in some examples and for a version of model 104, model data 112 may represent the architecture associated with model 104, one or more datasets used to train model 104, one or more risk scores associated with model 104, one or more licenses associated with model 104, one or more performance scores associated with model 104, the hardware on which model 104 is optimized to perform, one or more changes that have occurred regarding model 104, the identity of the user who performed the modifications, and / or the date and / or time of the modifications. In some examples, one or more modification components 110 may automatically generate, receive, obtain, and / or retrieve model data 112 during and / or after modifying model 104. In some examples, one or more modification components 110 may generate, receive, obtain, and / or retrieve model data 112 based on one or more events, such as a new version of model 104 being uploaded.

[0049] for example, Figure 3 Examples of obtaining data representing characteristics and / or modifications associated with a model, according to some embodiments of this disclosure, are illustrated. Figure 3In the example, one or more modified components 110 may at least obtain model data 302 associated with the sixth version 202 (6) of the model (the model data 302 may include and / or be similar to model data 112). As shown, the model data 302 may at least represent the architecture 304 of the sixth version 202 (6), one or more datasets 306 used to train the sixth version 202 (6), one or more risk scores 308 associated with the sixth version 202 (6), one or more licenses 310 associated with the sixth version 202 (6), one or more performance scores 312 associated with the sixth version 202 (6), the hardware on which the sixth version 202 (6) is optimized to be executed, and / or other information 316 associated with the characteristics and / or modifications of the sixth version 202 (6). In some examples, one or more modified components 110 can obtain similar model data associated with one or more other versions of the model 202(1)-(5).

[0050] Return to reference Figure 1 For example, the process 100 may include one or more information components 114 using at least a portion of the model data 112 to generate structured information 116 associated with model 104. As described herein, the structured information 116 associated with a version of model 104 may include, but is not limited to, information (e.g., messages) summarizing changes made (e.g., fine-tuning on a new dataset, quantization for inference using a specific type of processor, etc.), information describing details of modifications to model 104 (e.g., changes to architecture, changes to risk scores, changes to licenses, etc.), information describing performance improvements associated with model 104 (e.g., improved accuracy, improved accuracy with specific hardware, etc.), information describing the date the modifications occurred, information describing one or more users who performed the modifications, and / or information describing any other characteristics and / or modifications associated with model 104.

[0051] for example, Figure 4 Examples of generating structured information associated with a model version are illustrated according to some embodiments of the present disclosure. As shown, one or more information components 114 can at least process model data 302 associated with a sixth version 202 (6) of the model and / or model data 402 associated with one or more other versions 202 (1)-(5) of the model. Based at least on this processing, one or more information components 114 can generate structured information 404 associated with the sixth version 202 (6) of the model (the structured information 404 may include and / or be similar to structured information 116). Figure 4In the example, structured information 404 may include at least a submission message 406 indicating modifications, including fine-tuning the model on a new dataset; change details 408 including information about the new dataset; a performance improvement 410 indicating a 2% increase in model accuracy; and change details 412 indicating the user's identifier and the date and time the modification occurred. However, in other examples, structured information 404 may include additional and / or alternative types of information.

[0052] Return to reference Figure 1 For example, the process 100 may include one or more encoding components 118 encoding at least a portion of the structured information 116 to generate encoded information 120. As described herein, one or more encoding components 118 may include any type of encoder and / or use any type of encoder to generate the encoded information 120. For example, the encoded information 120 may include Unicode, ISO-8859-1, UTF-16, code pages, tag encoding, Base64 encoding, and / or any other type of encoded text. Although Figure 1 The example illustrates process 100 as including one or more encoding components 118 to generate encoded information 120, but in other examples, process 100 may not include one or more encoding components 118 to generate encoded information 120.

[0053] The process 100 may include one or more identifier components 122 processing at least a portion of the encoded information 120 (and / or, in some examples, structured information 116) to generate a unique identifier 124 associated with model 104, wherein the unique identifier 124 is stored in one or more databases 126. As described herein, the unique identifier 124 may include, but is not limited to, checksums, hashes, codes, letter identifiers, numeric identifiers, alphanumeric identifiers, and / or any other type of identifier that may be used to track modifications associated with model 104. Additionally, in some examples, one or more identifier components 122 may include one or more hash algorithms (e.g., SHA-256, Blake3, etc.) and / or use one or more hash algorithms (e.g., SHA-256, Blake3, etc.) to generate a checksum at least based on the encoded information 120. Furthermore, in some examples, one or more identifier components 122 may combine the encoded information 120 with additional information when generating the unique identifier 124. For example, one or more identifier components 122 can combine the encoded information 120 with the information describing the configuration 128 of the model 104, and then use the combined information to generate a unique identifier 124.

[0054] In some examples, one or more identification components 122 may generate unique identifiers 124 associated with various versions of model 104. For example, whenever a new version of model 104 is uploaded to one or more databases 106, one or more identification components 122 may generate unique identifiers 124 associated with the new version of model 104. However, in some examples, one or more identification components 122 may generate one or more unique identifiers 124 associated with one or more specific versions of model 104. For example, one or more identification components 122 may generate unique identifiers 124 for a specific type of modification, such as when model 104 is optimized for specific hardware.

[0055] for example, Figure 5 Examples of generating unique identifiers associated with various versions 202(1)-(6) of a model according to some embodiments of the present disclosure are illustrated. As shown, one or more identifier components 122 can generate at least a first unique identifier 502(1) associated with a first version (1) 202(1) of the model, a second unique identifier 502(2) associated with a second version 202(2) of the model, a third unique identifier 502(3) associated with a third version 202(3) of the model, a fourth unique identifier 502(4) associated with a fourth version 202(4) of the model, a fifth unique identifier 502(5) associated with a fifth version 202(5) of the model, and a sixth unique identifier 502(6) associated with a sixth version 202(6) of the model. 6) The associated sixth unique identifier 502(6). As described herein, unique identifiers 502(1)-(6) may be associated with features and / or modifications that are associated with versions 202(1)-(6) of the model. For example, the sixth unique identifier 502(6) may represent information associated with a fifth modification 204(5) of a fifth version 202(5) of the model to generate a sixth version 202(6) of the model. For example, one or more identifier components 122 may use structured information 404 to generate the sixth unique identifier 502(6).

[0056] Return to reference Figure 1For example, process 100 may include: locally storing model 104 and / or unique identifier 124 on one or more systems 102. For example, one or more systems may store unique identifier 124 in association with model 104, such that one or more systems 102 can later identify unique identifier 124 when one or more events occur. Additionally, in some examples, process 100 may include: providing at least some of the models in model 104 to external storage 130, wherein the model 104 provided to external storage 130 is represented by model 132; and / or providing at least some of the unique identifiers in unique identifier 124 to external storage 130, wherein the unique identifier 124 provided to external storage 130 is represented by unique identifier 134. In other words, one or more systems 102 may synchronize modifications to model 132 with external storage 130 while also providing the associated unique identifier 134 to external storage 130. In this way, external storage 130 may execute one or more of the processes described herein with respect to one or more systems 102.

[0057] As described herein, one or more systems 102, one or more computing devices 108, and / or external storage 130 may perform one or more tasks using a unique identifier 124 associated with model 104. For example, process 100 may include decoding the unique identifier 124 using one or more decoding components 136. As described herein, one or more decoding components 136 may decode the unique identifier 124 based on the occurrence of one or more events. For example, one or more decoding components 136 may decode the unique identifier 124 based on receiving a request from computing device 108, receiving the unique identifier 124 from computing device 104 (e.g., after providing the unique identifier 124 along with the associated model 104 to computing device 108), in response to providing the associated model 104 to computing device 108, and / or based on any other type of event. Additionally, one or more decoding components 136 may include any type of decoder and / or use any type of decoder to perform one or more processes described herein.

[0058] For example, based on a received request to decode a unique identifier 124 associated with model 104, one or more decoding components 136 can decode the unique identifier 124 to generate and / or retrieve structured information 116 associated with model 104. Additionally, in some examples, one or more decoding components 136 can generate one or more additional unique identifiers 124 associated with one or more additional versions of model 104. Thus, in such examples, one or more decoding components 136 are able to generate and / or retrieve structured information 116 representing clear audit trails associated with model 104. One or more systems 102 can then provide the structured information 116 associated with model 104 and / or the structured information 116 associated with one or more additional versions of model 104 to computing device 108. Based on the received structured information 116, computing device 108 can then provide the structured information 116 to a user (e.g., display the structured information 116, output audio associated with the structured information 116, etc.).

[0059] for example, Figure 6 Examples of providing structured information associated with various versions of a model according to some embodiments of the present disclosure are illustrated. As shown, user interface 602 may include at least first structured information 604(1) associated with a first modification 204(1) of a first version 202(1) of the model to generate a second version 202(2) of the model, second structured information 604(2) associated with a second modification 204(2) of the second version of the model 202(1) to generate a third version 202(3) of the model, and third structured information 604(3) associated with a third modification 204(3) of the third version 202(3) of the model to generate a fourth version 202(4) of the model. For example, the first structured information 604(1) indicates that the first modification 204(1) includes fine-tuning the model to generate task 1; the second structured information 604(2) indicates that the second modification 204(2) includes fine-tuning the model to generate task 2; and the third structured information 604(3) indicates that the third modification 204(3) includes optimizing the model for specific hardware (e.g., a specific type of processor, a specific type of memory, etc.).

[0060] Return to reference Figure 1In the example above, the user can then use the structured information 116 to perform one or more tasks. For the first example, if the user is determining whether to use a version of model 104 and / or is using a version of model 104, the user can use the structured information 116 to identify various modifications made to model 104, the performance of the version of model 104, the performance of the version of model 104 on specific hardware, the performance of the version of model 104 performing a specific task, the user making the modifications, the time of the modifications to model 104, and / or any other information important to the user. The user can then use this information to determine whether to use a version of model 104, such as performing one or more tasks and / or making further modifications.

[0061] In the second example, the user can use structured information 116 to identify the history of modifications that have occurred to model 104, such as what changes were made to generate each version of model 104. The user can then select the version of model 104 to use based on the structured information 116. For example, if the user determines that a modification to model 104 is accurate, sufficient, and / or necessary, the user can choose to perform one or more tasks using the latest version of model 104. However, if the user determines that one or more modifications are not accurate, sufficient, and / or necessary, the user can choose to revert to one or more modifications. Thus, the user can also use structured information 116 to identify versions of model 104 that do not include one or more modifications (such as versions of models to which one or more modifications have been performed) and use that version of the model to perform one or more tasks. In other words, the user can roll back to a version of model 104 prior to one or more modifications.

[0062] For example and Figure 6 In the example, the user can use user interface 602 to determine that the third modification 204(3) includes a model optimized for a specific hardware. However, the user may want to implement the model using different types of hardware. Thus, the user can use structured information 604(1)-(3) to determine a third version 202(3) using a model that has not yet been optimized for a specific hardware, rather than a fourth version 202(4) of a model optimized for a specific hardware.

[0063] In some examples, the unique identifier 124 can also be used to verify the integrity of model 104, such as by verifying that model 104 has not been tampered with and / or that model 104 has not undergone unauthorized modifications. For example, one or more systems may provide model 104 along with the associated unique identifier 124 to computing device 108. Computing device 108 can then use the unique identifier 124 to verify the integrity of model 104. For a first example, computing device 108 may compare the received unique identifier 124 with a unique identifier 123 stored on one or more systems 102. At least based on this comparison, when the unique identifier 124 matches, computing device 108 may verify the integrity of model 104; and / or when the unique identifier 124 does not match, model 104 is not verified. This is because in some examples, the unique identifier 124 may not match if model 104 has been further modified (e.g., tampered with, such as by performing unauthorized modifications).

[0064] In the second example, computing device 108 may perform a process similar to that of one or more systems 102 to generate a new unique identifier 124 associated with model 104. Computing device 108 may then again compare the received unique identifier 124 with the new unique identifier 124. Based at least on this comparison, computing device 108 may verify the integrity of model 104 when the unique identifier 124 matches; and / or not verify the integrity of model 104 when the unique identifier 124 does not match. This is because in some examples, when model 104 has been further modified (e.g., tampered with, such as by performing unauthorized modifications), the new unique identifier 124 may not match the received unique identifier 124. While these are just two example techniques for how computing device 108 may use the unique identifier 124 to verify the integrity of model 104, in other examples, computing device 108 may use additional and / or alternative techniques to use the unique identifier 124 to verify the integrity of model 104.

[0065] Although Figure 1The example illustrates one or more systems 102 as including one or more modification components 110, one or more information components 114, one or more encoding components 118, one or more identifier components 122, and one or more decoding components 136. However, in other examples, one or more computing devices 108 and / or external storage 130 may include one or more modification components 110, one or more information components 114, one or more encoding components 118, one or more identifier components 122, and one or more decoding components 136. For example, process 100 can be performed using one or more systems 102, one or more computing devices 108, and / or external storage 130. Additionally, components (e.g., one or more modification components 110, one or more information components 114, one or more encoding components 118, one or more identifier components 122, and one or more decoding components 136) may include software, hardware, machine learning models, neural networks, modules, classifiers, algorithms, encoders, decoders, programs, applications, and / or any other type of processing component.

[0066] Figure 7 An example of a process 700 for automatically updating a model card to include supplemental information on modifications to the model, according to some embodiments of this disclosure, is illustrated. It should be understood that the arrangements and other arrangements described herein are illustrative only. Other arrangements and elements (e.g., machines, interfaces, functions, sequences, functional groupings, etc.) may be used in addition to or as alternatives to the arrangements and elements shown, and some elements may be omitted entirely. Furthermore, many of the elements described herein are functional entities that can be implemented as discrete or distributed components, or in combination with other components, and in any suitable combination and location. The various functions described herein as being performed by entities can be performed by hardware, firmware, and / or software. For example, various functions can be performed using a processor that executes instructions stored in memory.

[0067] For example, the process 700 may include one or more detection components 702 for determining that model 104 is being modified, model 104 has been modified, a new version of model 104 has been generated, and / or the new version of model 104 is stored in one or more databases 106. In some examples, to determine that model 104 has been modified, one or more detection components 702 may use a unique identifier 124 associated with the new version of model 104. For the first example, one or more detection components 702 may determine that the unique identifier 124 is being (e.g., by using data from...) Figure 1The example process 100) is generated and / or newly generated. For example, and as described herein, when model 104 is modified, a unique identifier 124 for model 104 can be generated to track and / or verify modifications that occur to model 104.

[0068] In the second example, one or more detection components 702 can determine that unique identifier 124 is new for one or more databases 126 and / or different from one or more other unique identifiers 124 associated with one or more previous versions of model 104. For example, and as described herein, each unique identifier 124 is different from other unique identifiers 124 because it is generated using information describing the characteristics associated with model 104 and / or modifications that have occurred for model 104. Thus, if one or more detection components 702 detect a new unique identifier 124 within one or more databases 126, one or more detection components 702 can determine that model 104 associated with unique identifier 124 has been modified. While these are just two example techniques for one or more detection components 702 to determine that model 104 has been modified using unique identifier 124, in other examples, one or more detection components 702 may use additional and / or alternative techniques.

[0069] In some examples, one or more detection components 702 may use one or more additional and / or alternative techniques to determine that model 104 is being modified and / or has been modified. For example, one or more detection components 104 may determine that model 104 is being modified and / or has been modified based on: determining that computing device 108 is accessing model 104, determining that computing device 108 is modifying model 104, determining that computing device 108 is uploading a new version of model 104 to one or more systems 102, receiving an indication that model 104 is being modified, receiving an indication that model 104 has been modified, receiving an indication that a new version of model 104 exists, and / or using any other techniques.

[0070] Then, the process 700 may include: one or more decoding components 704 processing the unique identifier 124 to obtain modification information 706 associated with the modification, wherein the modification information 706 may also be referred to as supplementary information. For example, based on determining that model 104 has been modified to generate a new version of model 104, one or more decoding components 704 may encode the unique identifier 124 associated with model 104 to retrieve structured information 116 describing the characteristics and / or modifications associated with model 104, wherein the modification information 706 may include at least a portion of the structured information 116. As described herein, one or more decoding components 704 may include any type of decoder and / or use any type of decoder to perform one or more processes in the process described herein. Thus, the process 700 may include: determining that model 104 has been modified using the unique identifier 124 associated with model 104, and then further using the unique identifier 124 to obtain modification information 706 describing the modifications associated with model 104.

[0071] for example, Figure 8 Examples of decoding unique identifiers according to some embodiments of this disclosure to obtain supplementary information associated with modifications to the model are illustrated. Figure 8 In one example, one or more detection components 702 may first determine that the model has been modified using a sixth unique identifier 502(6) associated with the sixth version 202(6) of the model. Thus, one or more decoding components 704 may decode the sixth unique identifier 502(6) to obtain modification information 802 describing a fifth modification 204(5) that has occurred with respect to the sixth version 202(6) of the model. For example, modification information 802 may include at least a submission message 406, change details 408, performance improvements 410, and / or modification details 412 from the structured information 404 associated with the sixth version 202(6) of the model. However, in other examples, modification information 802 may include additional and / or alternative information associated with the fifth modification 204(5) of the sixth version of the model 202(6).

[0072] Return to reference Figure 7For example, the process 700 may include one or more test components 708 for processing model 104 to generate performance information 710 associated with model 104, wherein the performance information 710 may also be referred to as supplementary information. For instance, based on the determination that model 104 has been modified, one or more test components 708 may perform one or more tests on the new version of model 104 to determine performance information 710. As described herein, the performance information 710 associated with model 104 may include, but is not limited to, accuracy score, precision score, recall score, loss score, throughput, latency, memory utilization, and / or any other type of performance metric associated with model 104. Additionally, one or more test components 708 may perform any type of test on model 104 to determine performance information 710.

[0073] For example, one or more test components 708 may input instances of input data into model 104 for processing. Instances of input data may include images (if model 104 includes a visual model), audio data representing speech (if model 104 includes a language model 104), and / or any other type of input data corresponding to the type of model 104. One or more test components 708 may then process the input data based at least on model 104 to obtain outputs and use these outputs to determine performance information 710 associated with model 104. For example, one or more test components 708 may determine at least an accuracy score associated with model 104 by comparing the output of model 104 with a truth output associated with the input data. While this is only one example technique of how one or more test components 708 can test model 104 to determine performance information 710, in other examples, one or more test components 708 may use additional and / or alternative techniques.

[0074] for example, Figure 9An example of a test model for determining performance information according to some embodiments of the present disclosure is illustrated. As shown, one or more test components 708 may input data 902 into a sixth version 202 (6) of the model. As described herein, the input data 902 may be associated with the type of data that the sixth version 202 (6) of the model was trained to process. For example, the input data 902 may include image data, video data, audio data, text data, sensor data, and / or any other type of data. The sixth version 202 (6) of the model may then process the input data 902 and generate output data 904 based at least on the processing. One or more test components 708 may then use the output data 904 together with truth data 906 corresponding to the input data 902 to determine performance information 908 associated with the sixth version 202 (6) of the model (which performance information 908 may include and / or be similar to performance information 710).

[0075] For example, one or more test components 708 may determine an accuracy score associated with the sixth version 202(6) of the model based at least on comparing output data 904 with truth data 906. For example, one or more test components 708 may determine an accuracy score based on the number of correct outputs and the total number of outputs, as determined by comparing output data 904 with truth data 906. Additionally, in some examples, one or more test components 708 may determine additional performance information 908 associated with the sixth version 202(6) of the model based at least on testing, such as accuracy score, recall score, loss score, throughput, latency, memory utilization, and / or any other type of performance metric associated with the sixth version 202(6).

[0076] Return to reference Figure 7 For example, the process 700 may include one or more update components 712 that receive at least modification information 706 and performance information 710. In some examples, one or more update components 712 may receive additional supplementary information associated with modifications to model 104, such as information from... Figure 1 The example model data 112 represents the information. The process 100 can then include one or more update components 712 that update at least partially the model card 714 associated with model 104 using supplementary information, wherein model card 714 is stored in one or more databases 716. As described herein, one or more update components 712 can update model card 714 using supplementary information.

[0077] For example, model card 714 may initially include fields providing initial information associated with the first version of model 104, such as attributes, model architecture, intended use cases, extended applications, inputs, outputs, intended users, model performance for different groups, training, constraints, computational requirements, etc. Additionally, attributes may include, but are not limited to, the name and / or identifier of model 104, the names and / or identifiers of one or more datasets used to train model 104, the size of one or more datasets, the number of training epochs, the license type associated with model 104, one or more risk scores associated with model 104, one or more bias scores associated with model 104, one or more losses associated with model 104, and / or any other type of attribute. In some examples, one or more systems 102 may automatically generate model card 714 upon receiving the first version of model 104. In some examples, one or more systems 102 may receive model card 714 along with the first version of model 104.

[0078] Then, one or more update components 712 may update model card 714 to include supplemental information 104 associated with modifications made to model 104. For example, one or more update components 712 may modify model card 714 to include additional fields representing information related to the modification, such as the type of modification, a unique identifier 124 associated with model 104, the date of the modification, performance information associated with the modification (e.g., accuracy score, precision score, recall score, etc.), the date the modified model 104 was deployed, the hardware on which model 104 was optimized, performance characteristics associated with model 104 implemented on the hardware (e.g., throughput, latency, power consumption, memory utilization, etc.), and / or any other information associated with the modification. In some examples, one or more update components 712 may update model card 714 for each modification performed on model 104.

[0079] For example, one or more update components 712 may receive supplementary information associated with a new version of the model, such as modification information 706, performance information 710, and / or information represented by model data 112. One or more update components 712 may also receive an identifier associated with model 104 and / or an identifier associated with model card 714 corresponding to model 104. One or more update components 712 may then use one or more identifiers to retrieve model card 714 associated with model 104. In some examples, model card 714 may include initial information associated with model 104. In some examples, model card 714 may additionally include supplementary information associated with one or more previous modifications to model 104. In any of these examples, one or more update components 712 may then use at least a portion of the supplementary information to update model card 714 to include information describing a new version of the modified model 104.

[0080] for example, Figures 10A-10C Examples of updated model cards according to some embodiments of this disclosure, to include supplementary information associated with modifications to the model, are illustrated. Figure 10A As shown in the example, model card 1002 (which may include and / or be similar to model card 714) may initially include information 1004 corresponding to a first version 202(1) of the model. For example, information 1004 may at least describe the model version 1006, the model's unique identifier 1008 (e.g., name, etc.), the architecture associated with the model 1010, one or more datasets 1012 used to initially train the model, and the model's performance 1014. However, in other examples, information 1004 may include additional and / or alternative fields describing additional and / or alternative characteristics associated with the model.

[0081] Then and as Figure 10BAs shown in the example, one or more update components 712 may then receive supplementary information 1016 associated with a first modification 204(1) of a first version 202(1) of the model to generate a second version 202(2) of the model. One or more update components 712 may then use at least a portion of the supplementary information 1016 to update the model card 1002 to describe the first modification 204(1) and / or the second version 202(2) of the model. For example and as shown, the supplementary information 1016 may at least describe version 1018, modification characteristics 1020 (e.g., type of modification, date of modification, updated accuracy associated with the modification, deployment date, etc.), deployment characteristics 1022 (e.g., type of hardware, throughput, latency, memory utilization, etc.), performance 1024 (e.g., accuracy score, precision score, recall score, etc.) and a unique identifier 1026 associated with the second version 202(2) of the model (e.g., a second unique identifier 502(2)). However, in other examples, supplementary information 1016 may include additional and / or alternative fields describing additional dirt and / or alternative properties associated with the model.

[0082] Then and as Figure 10C As shown in the example, one or more update components 712 may then receive supplementary information 1028 associated with a second modification 204(2) of a second version 202(2) of the model to generate a third version 202(3) of the model. One or more update components 712 may then use at least a portion of the supplementary information 1028 to update the model card 1002 to describe the second modification 204(2) and / or the third version 202(3) of the model. For example and as shown, the supplementary information 1028 may at least describe version 1030, modification characteristics 1032 (e.g., type of modification, date of modification, updated accuracy associated with the modification, deployment date, etc.), deployment characteristics 1034 (e.g., type of hardware, throughput, latency, memory utilization, etc.), performance 1036 (e.g., accuracy score, precision score, recall score, etc.) and a unique identifier 1038 associated with the third version of the model 202(3) (e.g., a third unique identifier 502(3)). However, in other examples, supplementary information 1028 may include additional and / or alternative fields describing additional and / or alternative characteristics associated with the model.

[0083] Then, the process can continue to repeat in order to update model card 1002 with supplementary information associated with the third modification 204(3), the fourth version 202(4) of the model, the fourth modification 204(4), the fifth version 202(5) of the model, the fifth modification 204(5) and / or the sixth version 202(5) of the model. Although Figures 10A-10CThe example describes updating model card 1002 to include supplemental information associated with all modifications 204(1)-(5) and / or all modified versions 202(2)-(6) of the model, but in other examples, one or more update components 712 may generate and / or update multiple model cards associated with the model. For example, one or more update components 108 may generate a first model card including information associated with modifications 204(1)-(3) and versions 202(1)-(4) of the model and a second model card associated with modifications 204(4)-(5) and versions 202(1) and 202(5)-(6) of the model.

[0084] Return to reference Figure 7 For example, the process 700 may include: one or more update components 712 outputting updated model cards 718 associated with model 104. In some examples, one or more systems 102 may then store the updated model cards 718 associated with model 104 (in, for example, one or more databases 716). In some examples, one or more systems 102 may also send at least some of the updated model cards 718 to an external storage library 130 for storage, wherein the updated model cards 718 sent to the external storage library 130 may be represented by model cards 720. In other words, one or more systems 102 may synchronize updates to the updated model cards 718 with the external storage library 130.

[0085] In some examples, one or more systems 102 may then perform one or more operations using the updated model card 718. For example, and for model 104, one or more systems 102 may generate a user interface 722 that includes at least a portion of the updated model card 718 associated with model 104. As described herein, one or more systems 102 may generate the user interface 722 in response to one or more events, such as receiving a request for model 104, receiving a request for a version of model 102, receiving a request for information about model 104, receiving a request for information about a version of model 104, and / or any other event. One or more systems 102 may then send data representing the user interface 722 to at least one computing device 108. In this way, the computing device 108 may provide the user interface 722 to one or more users who can use the user interface 722 to identify information associated not only with the initial version of model 104 but also with any modified version of model 104.

[0086] For example, one or more users can use user interface 722 to identify specific modifications made to model 104. In the first example, if one or more users are searching for one or more versions of model 104 modified to perform a specific task (such as object detection for an object type), then one or more users can use user interface 722 to identify the version of model 104 trained to perform the specific task. In the second example, if one or more users are searching for one or more versions of model 104 optimized for implementation on specific hardware (such as a processor), then one or more users can use user interface 722 to identify the version of model 104 optimized for the specific hardware. However, in the third example, if one or more users are searching for one or more versions of model 104 that include specific performance requirements (such as an accuracy score equal to or greater than 95%), then one or more users can use user interface 722 to identify the version of model 104 that meets the specific performance requirements.

[0087] As described herein, in some examples, one or more systems 102 may provide updated model cards 718 in various formats. For a first example, one or more systems 102 may generate updated model cards 718 in a text format, such that user interface 722 presents a text file corresponding to the updated model card 718. In this example, the updated model card 718 may still be searchable, such as by allowing users to input text queries, providing interface elements (e.g., labels, buttons, etc.) that can be selected to move to different parts of the updated model card 718, and / or using any other techniques. For a second example, one or more systems 102 may generate updated model cards 718 in an interactive format, such that user interface 722 allows users to interact with the updated model card 718. For example, a user may be able to select different parts of the updated model card 718, such as parts corresponding to supplementary information about different modifications to model 104. Based on the selection of a part of the updated model card 718, additional information corresponding to the selected part may then be provided to the user.

[0088] Although Figure 7The example illustrates one or more systems 102 as including one or more detection components 702, one or more decoding components 704, one or more test components 708, and one or more update components 712. However, in other examples, one or more computing devices 108 and / or external storage 130 may include one or more detection components 702, one or more decoding components 704, one or more test components 708, and one or more update components 712. For example, process 700 may be performed using one or more systems 102, one or more computing devices 108, and / or external storage 130. Additionally, components (e.g., one or more detection components 702, one or more decoding components 704, one or more test components 708, and one or more update components 712) may include software, hardware, machine learning models, neural networks, modules, classifiers, algorithms, encoders, decoders, programs, applications, and / or any other type of processing component.

[0089] Now, for reference Figure 11-15 Each block of methods 1100, 1200, 1300, 1400, and 1500 described herein includes a computational process that can be executed using any combination of hardware, firmware, and / or software. For example, various functions can be performed by a processor executing instructions stored in memory. These methods 1100, 1200, 1300, 1400, and 1500 can also be embodied as computer-usable instructions stored on a computer storage medium. These methods 1100, 1200, 1300, 1400, and 1500 can be provided by standalone applications, services (standalone or combined with another managed service), managed services, or plug-ins to another product, to name a few. Additionally, by way of example, regarding Figure 1 and Figure 7 The system describes methods 1100, 1200, 1300, 1400, and 1500. However, these methods 1100, 1200, 1300, 1400, and 1500 may be additionally or alternatively implemented by any one of the systems or any combination of systems, including but not limited to the systems described herein.

[0090] Figure 11The illustration shows a flowchart of a method 1100 for generating a unique identifier associated with modifications to a model, according to some embodiments of the present disclosure. At block 1102, the method 1100 may include: obtaining data representing one or more modifications that have occurred with respect to a first version of the model for generating a second version of the model. For example, one or more systems 102 (e.g., one or more modification components 110) may obtain data 112 representing one or more modifications that have occurred with respect to a first version of model 104 for generating a second version of model 104. As described herein, one or more users may perform one or more modifications using one or more computing devices 108. Additionally, one or more modifications may include: fine-tuning model 104 to perform a specific task, optimizing model 104 to perform using specific hardware components (e.g., processor, memory, etc.), hyperparameter tuning of model 104, quantization of model 104, and / or performing any other type of model modification technique.

[0091] At box B1104, method 1100 may include: generating information associated with at least one or more modifications, based at least on data. For example, one or more systems 102 (e.g., one or more information components 114) may use model data 112 to generate structured information 116. As described herein, structured information 116 may include details describing at least one or more modifications to a first version of model 104 for generating a second version of model 104. For example, structured information 116 may include, but is not limited to, information summarizing the changes made, information describing details of one or more modifications to model 104, information describing performance improvements associated with model 104, information describing the dates on which one or more modifications occurred, information describing the users who performed one or more modifications, and / or information describing any other characteristics and / or modifications associated with model 204.

[0092] At box B1106, method 1100 may include: generating a unique identifier associated with a second version of the model, at least based on the information stated herein. For example, one or more systems 102 (e.g., one or more encoding components 118 and / or one or more identifier components 122) may generate a unique identifier 124 using at least the structured information 116. As described herein, one or more systems 102 may generate the unique identifier 124 by: first, encoding the structured information 116 to generate encoded information 120; and then computing the unique identifier 124 using one or more hash algorithms. Additionally, in the example, one or more systems 102 may first combine the encoded information 120 with the model configuration 128 before computing the unique identifier 124 using one or more hash algorithms. In some examples, the unique identifier may include a checksum. However, in other examples, the unique identifier may include any other type of identifier.

[0093] At box B1108, method 1100 may include performing one or more operations using a unique identifier. For example, one or more systems 102 may perform one or more operations using unique identifier 124. As described herein, one or more operations may include, but are not limited to, storing unique identifier 124 in association with a second version of model 104, providing unique identifier 124 to external storage 130, decoding unique identifier 124 to provide structured information 116 to one or more users, verifying the integrity of the second version of model 104, and / or any other operation.

[0094] Figure 12 The illustration shows a flowchart of a method 1200 for providing information associated with a model using a unique identifier, according to some embodiments of the present disclosure. At block B1202, the method 1200 may include: receiving a request associated with the model. For example, one or more systems 102 may receive a request associated with a model 104 from a computing device 108. In some examples, the request may be associated with: accessing and / or using the model 104, retrieving information associated with the model 104, retrieving information associated with multiple versions of the model 104, decoding the unique identifier 124 associated with the model 104, and / or any other type of request.

[0095] At box B1204, method 1200 may include: identifying a unique identifier associated with the model. For example, one or more systems 102 may (e.g., from one or more databases 126) identify a unique identifier 124 associated with model 104. In some examples, one or more systems 102 may also identify additional unique identifiers 124 associated with one or more additional versions of model 104.

[0096] At box B1206, method 1200 may include: determining information associated with one or more modifications to the model, at least based on a unique identifier. For example, one or more systems 102 may then use the unique identifier 124 to determine structured information 116 associated with model 104. For example, one or more systems 102 may determine the structured information 116 by decoding the unique identifier 124. Additionally, in some examples, one or more systems may use one or more additional unique identifiers 124 to determine additional structured information 116 associated with one or more additional versions of model 104.

[0097] At box B1208, method 1200 may include providing information associated with one or more modifications. For example, one or more systems 102 may provide structured information 116 associated with model 104. Additionally, in some examples, one or more systems 102 may provide structured information 116 associated with one or more additional versions of model 104. In some examples, providing structured information 116 may include sending structured information 116 to computing device 108 for display by computing device 108.

[0098] Figure 13 The illustration shows a flowchart of a method 1300 for verifying a model using a unique identifier according to some embodiments of the present disclosure. At block B1302, the method 1300 may include receiving a model and a unique identifier associated with the model. For example, computing device 108 (and / or external storage 130) may receive a model 104 and a unique identifier 124 associated with the model 104. As described herein, in some examples, the unique identifier 124 may include a checksum generated using structured information 116 associated with the model 104 and one or more hash algorithms. However, in other examples, the unique identifier 124 may include any other type of identifier that can be used to verify the integrity of the model 104.

[0099] At box B1304, method 1300 may include comparing a unique identifier and an additional unique identifier associated with the model. For example, computing device 108 (and / or external storage 130) may compare the received unique identifier 124 and the additional unique identifier 124. In some examples, the additional unique identifier 124 may be stored by one or more systems 102. In some examples, the additional unique identifier 124 may be generated by computing device 108 (such as through a similar process of using one or more systems 102 to generate the received unique identifier 124).

[0100] At box B1306, method 1300 may include: determining whether the integrity of the model has been verified, at least based on the comparison. For example, computing device 108 (and / or external storage 130) may determine whether the integrity of model 104 has been verified, at least based on the comparison. For example, computing device 108 may determine that the integrity of model 104 has been verified when the unique identifier 124 matches, or determine that the integrity of model 104 has not been verified when the unique identifier 124 does not match.

[0101] Figure 14 The illustration shows a flowchart of a method 1400 for updating a model card using information describing modifications to a model, according to some embodiments of the present disclosure. At block B1402, the method 1400 may include: determining one or more modifications that have occurred to a first version of the model for generating a second version of the model. For example, one or more systems 102 (e.g., one or more detection components 702) may determine one or more modifications that have occurred to a first version of model 104 for generating a second version of model 104. As described herein, in some examples, one or more systems 102 may use a unique identifier 124 associated with the second version of model 104 to determine that one or more modifications have occurred. In some examples, one or more systems 102 may use another technique (such as one or more systems 102 providing a first version of model 104 for modification, one or more systems 102 receiving a second version of model 104, one or more systems receiving an indication that one or more modifications have occurred, and / or any other technique) to determine that one or more modifications have occurred.

[0102] At box B1404, method 1400 may include determining information relating to the one or more modifications. For example, one or more systems 102 may determine supplementary information relating to the one or more modifications. In some examples, one or more systems 102 (e.g., one or more decoding components 704) may determine at least a portion of the supplementary information (such as modification information 706) based at least on the decoding of the unique identifier 124. In some examples, one or more systems 102 (e.g., one or more testing components 708) may determine at least a portion of the supplementary information (such as performance information 710) based at least on testing a second version of model 104. However, in other examples, one or more systems 102 may use any other technique to determine the supplementary information relating to the one or more modifications.

[0103] At box B1406, method 1400 may include: obtaining a model card associated with the first version of the model. For example, one or more systems 102 (e.g., one or more update components 712) may obtain a model card 714 associated with the first version of model 104. In some examples, one or more systems 102 may obtain model card 714 and / or an identifier associated with model card 714 using an identifier associated with the first version of model 104. However, in other examples, one or more systems 102 may obtain model card 714 using any other technique.

[0104] At box B1408, method 1400 may include: generating an updated model card by updating the model card to include at least a portion of the information. For example, one or more systems 102 (e.g., one or more update components 712) may update model card 714 using at least a portion of the supplementary information to generate an updated model card 718. In some examples, one or more systems 102 may update model card 714 by adding text to model card 714 corresponding to at least a portion of the supplementary information. In some examples, this may continue to be repeated such that the updated model card 718 is updated with supplementary information relating to one or more additional modifications to model 104.

[0105] At block B1410, method 1400 may include: determining, at least based on the updated model card and one or more capabilities associated with the computing device, a second version of the model to be provided to the computing device. For example, one or more systems 102 may obtain information indicating one or more capabilities associated with the computing device 108. As described herein, these capabilities may include, but are not limited to, processing power, memory capacity, networking capabilities, and / or any other type of capability. One or more systems 102 may then use the updated model card 718 to determine that the computing device 108 is capable of implementing the second version of model 104 based on one or more capabilities. For example, one or more systems 102 may determine that the one or more capabilities satisfy one or more minimum capabilities for implementing the second version of model 104.

[0106] At box B1412, method 1400 may include sending data to the computing device for implementing the second version of the model. For example, one or more systems 102 may send data to computing device 108 for implementing the second version of model 104. In some examples, one or more systems 102 send data at least based on determining that computing device 108 is capable of implementing the second version of model 104.

[0107] Figure 15 The illustration shows a flowchart of another method 1500 for updating a model card by means of information describing modifications to the model, according to some embodiments of the present disclosure. At block B1502, the method 1500 may include: determining information relating to one or more modifications to the model. For example, one or more systems 102 (e.g., one or more detection components 702) may determine that model 104 has been modified to generate a new version of model 104. Then, one or more systems 102 may determine supplementary information relating to one or more modifications. In some examples, one or more systems 102 (e.g., one or more decoding components 704) may determine at least a portion of the supplementary information (such as modification information 706) based at least on the decoding of a unique identifier 124. In some examples, one or more systems 102 (e.g., one or more testing components 708) may determine at least a portion of the supplementary information (such as performance information 710) based at least on testing a second version of model 104. However, in other examples, one or more systems 102 may use any other techniques to determine the supplementary information relating to one or more modifications.

[0108] At box B1504, method 1500 may include: generating an updated model card by updating the model card to include at least a portion of the information. For example, one or more systems 102 (e.g., one or more update components 712) may update model card 714 using at least a portion of the supplementary information to generate an updated model card 718. In some examples, one or more systems 102 may update model card 714 by adding text to model card 714 corresponding to at least a portion of the supplementary information. In some examples, this may continue to be repeated such that the updated model card 718 is updated with additional supplementary information relating to one or more additional modifications to model 104.

[0109] At box B1506, method 1500 may include storing an updated model card in association with a modified model. For example, one or more systems 102 may store an updated model card 718 in association with a new version of model 104. In some examples, one or more systems 102 may also synchronize a new version of model 104 and / or updated model card 718 with one or more external repositories 130 (e.g., by sending the updated model card 718 and / or the new version of model 104 to one or more external repositories 130).

[0110] Example computing device

[0111] Figure 16 This is a block diagram of an example computing device 1600 suitable for implementing at least some embodiments of the present disclosure. The computing device 1600 may include an interconnect system 1602 directly or indirectly coupled to the following devices: memory 1604, one or more central processing units (CPUs) 1606, one or more graphics processing units (GPUs) 1608, a communication interface 1610, input / output (I / O) ports 1612, input / output components 1614, a power supply 1616, one or more presentation components 1618 (e.g., displays), and one or more logic units 1620. In at least one embodiment, the computing device 1600 may include one or more virtual machines (VMs), and / or any component thereof may include virtual components (e.g., virtual hardware components). For a non-limiting example, one or more GPUs 1608 may include one or more vGPUs, one or more CPUs 1606 may include one or more vCPUs, and / or one or more logic units 1620 may include one or more virtual logic units. Therefore, computing device 1600 may include discrete components (e.g., a complete GPU dedicated to computing device 1600), virtual components (e.g., a portion of the GPU dedicated to computing device 1600), or a combination thereof.

[0112] although Figure 16 The various blocks are shown connected via an interconnect system 1602 with wiring, but this is not intended to be limiting and is merely for clarity. For example, in some embodiments, a presentation component 1618, such as a display device, may be considered an I / O component 1614 (e.g., if the display is a touchscreen). As another example, the CPU 1606 and / or GPU 1608 may include memory (e.g., memory 1604 may represent a storage device other than the memory of the GPU 1608, CPU 1606, and / or other components). Therefore, Figure 16 The computing devices mentioned are merely illustrative. No distinction is made between categories such as "workstation," "server," "laptop," "desktop," "tablet," "client device," "mobile device," "handheld device," "game console," "electronic control unit (ECU)," "virtual reality system," and / or other device or system types, as all of these are considered within the same category. Figure 16 Within the scope of computing devices.

[0113] Interconnect system 1602 may represent one or more links or buses, such as address buses, data buses, control buses, or combinations thereof. Interconnect system 1602 may include one or more bus or link types, such as Industry Standard Architecture (ISA) bus, Extended Industry Standard Architecture (EISA) bus, Video Electronics Standards Association (VESA) bus, Peripheral Component Interconnect (PCI) bus, Peripheral Component Interconnect Fast (PCIe) bus, and / or another type of bus or link. In some embodiments, there is a direct connection between components. For example, CPU 1606 may be directly connected to memory 1604. Furthermore, CPU 1606 may be directly connected to GPU 1608. Where there is a direct or point-to-point connection between components, interconnect system 1602 may include a PCIe link to perform the connection. In these examples, a PCI bus is not required in computing device 1600.

[0114] The memory 1604 may include any of a wide variety of computer-readable media. Computer-readable media can be any available medium that can be accessed by the computing device 1600. Computer-readable media may include volatile and non-volatile media, as well as removable and non-removable media. For example and without limitation, computer-readable media may include computer storage media and communication media.

[0115] Computer storage media may include volatile and non-volatile media and / or removable and non-removable media, implemented in any way or by any method or technique for storing information such as computer-readable instructions, data structures, program modules, and / or other data types. For example, memory 1604 may store computer-readable instructions (e.g., representing programs and / or program elements, such as an operating system). Computer storage media may include, but is not limited to, RAM, ROM, EEPROM, flash memory or other storage technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage devices, magnetic tape cassettes, magnetic tape, disk storage devices or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by computing device 1600. As used herein, computer storage media does not include the signal itself.

[0116] Computer storage media may include computer-readable instructions, data structures, program modules, and / or other data types in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium. The term "modulated data signal" may refer to a signal whose characteristics are set or altered in a manner that encodes information into that signal. For example and without limitation, computer storage media may include wired media such as wired networks or direct wired connections, and wireless media such as sound, RF, infrared, and other wireless media. Any combination of the above should also be included within the scope of computer-readable media.

[0117] CPU 1606 may be configured to execute at least some of computer-readable instructions to control one or more components of computing device 1600 to perform one or more of the methods and / or processes described herein. Each of CPU 1606 may include one or more cores (e.g., one, two, four, eight, twenty-eight, seventy-two, etc.) capable of processing a large number of software threads simultaneously. CPU 1606 may include any type of processor and may include different types of processors depending on the type of computing device 1600 implemented (e.g., processors with fewer cores for mobile devices and processors with more cores for servers). For example, depending on the type of computing device 1600, the processor may be an advanced RISC machine (ARM) processor implemented using Reduced Instruction Set Computing (RISC) or an x86 processor implemented using Complex Instruction Set Computing (CISC). In addition to one or more microprocessors or supplementary coprocessors such as math coprocessors, computing device 1600 may also include one or more CPUs 1606.

[0118] In addition to or as a replacement for CPU 1606, one or more GPUs 1608 may be configured to execute at least some of computer-readable instructions to control one or more components of computing device 1600 to perform one or more of the methods and / or processes described herein. One or more GPUs 1608 may be integrated GPUs (e.g., having one or more CPUs 1606) and / or one or more GPUs 1608 may be discrete GPUs. In embodiments, one or more GPUs 1608 may be coprocessors of one or more CPUs 1606. Computing device 1600 may use GPUs 1608 to render graphics (e.g., 3D graphics) or perform general-purpose computing. For example, one or more GPUs 1608 may be used for general-purpose computing on a GPU (GPGPU). One or more GPUs 1608 may include hundreds or thousands of cores capable of processing hundreds or thousands of software threads simultaneously. GPUs 1608 may generate pixel data for outputting an image in response to rendering commands (e.g., rendering commands received from CPU 1606 via a host interface). GPU 1608 may include graphics memory, such as display memory, for storing pixel data or any other suitable data, such as GPGPU data. Display memory may be included as part of memory 1604. One or more GPUs 1608 may include two or more GPUs operating in parallel (e.g., via a link). The link may be directly connected to the GPUs (e.g., using NVLINK) or connected via a switch (e.g., using NVSwitch). When combined, each GPU 1608 may generate pixel data or GPGPU data for different portions of the output or different outputs (e.g., the first GPU for the first image, the second GPU for the second image). Each GPU may include its own memory or may share memory with other GPUs.

[0119] In addition to or as an alternative to CPU 1606 and / or GPU 1608, logic unit 1620 may be configured to execute at least some of computer-readable instructions to control one or more components of computing device 1600 to perform one or more of the methods and / or processes described herein. In embodiments, CPU 1606, GPU 1608, and / or logic unit 1620 may execute any combination of methods, processes, and / or portions thereof, discretely or jointly. One or more logic units 1620 may be part of and / or integrated into one or more of CPU 1606 and / or GPU 1608, and / or one or more logic units 1620 may be discrete components or otherwise separate from CPU 1606 and / or GPU 1608. In embodiments, one or more logic units 1620 may be coprocessors of one or more CPUs 1606 and / or one or more GPUs 1608.

[0120] Examples of logic unit 1620 include one or more processing cores and / or components thereof, such as data processing unit (DPU), tensor core (TC), tensor processing unit (TPU), pixel vision core (PVC), vision processing unit (VPU), graphics processing cluster (GPC), texture processing cluster (TPC), streaming multiprocessor (SM), tree traversal unit (TTU), artificial intelligence accelerator (AIA), deep learning accelerator (DLA), programmable vision accelerator (PVAs) — which may include one or more direct memory access (DMA) systems, one or more vision or vector processing units (vpu), one or more pixel processing engines (PPE) — for example, including a 2D array of processing elements, each processing element communicating east, south, east, and west with one or more other processing elements in the array, one or more decoupling accelerators or units (e.g., decoupling lookup table (DLUT) accelerators or units), etc. Visual processing unit (VPU), optical flow accelerator (OFA), field-programmable gate array (FPGA), neuromorphic chip, quantum processing unit (QPU), associative processing unit (APU), arithmetic logic unit (ALU), application-specific integrated circuit (ASIC), floating-point unit (FPU), input / output (I / O) element, peripheral component interconnect (PCI) or peripheral component interconnect fast (PCIe) element, etc.

[0121] Communication interface 1610 may include one or more receivers, transmitters, and / or transceivers that allow computing device 1600 to communicate with other computing devices via electronic communication networks, including wired and / or wireless communications. Communication interface 1610 may include components and functions that allow communication via any of several different networks, such as wireless networks (e.g., Wi-Fi, Z-Wave, Bluetooth, Bluetooth LE, ZigBee, etc.), wired networks (e.g., communication via Ethernet or InfiniBand), low-power wide area networks (e.g., LoRaWAN, SigFox, etc.), and / or the Internet. In one or more embodiments, logic unit 1620 and / or communication interface 1610 may include one or more data processing units (DPUs) to directly transmit data received via a network and / or via interconnect system 1602 to one or more GPUs 1608 (e.g., their memory).

[0122] I / O port 1612 allows computing device 1600 to be logically coupled to other devices, including I / O component 1614, presentation component 1618, and / or other components, some of which may be built into (e.g., integrated into) computing device 1600. Illustrative I / O component 1614 includes microphones, mice, keyboards, joysticks, game pads, game controllers, satellite dish antennas, scanners, printers, wireless devices, and so on. I / O component 1614 can provide a Natural User Interface (NUI) for processing user-generated air gestures, voice, or other physiological input. In some instances, the input may be transmitted to appropriate network elements for further processing. The NUI can implement any combination of voice recognition, stylus recognition, facial recognition, biometric recognition, on-screen and adjacent-screen gesture recognition, air gestures, head and eye tracking, and touch recognition associated with the display of computing device 1600 (described in more detail below). Computing device 1600 may include depth cameras such as stereo camera systems, infrared camera systems, RGB camera systems, touchscreen technology, and combinations thereof for gesture detection and recognition. In addition, computing device 1600 may include an accelerometer or gyroscope that allows motion detection (e.g., as part of an inertial measurement unit (IMU)). In some examples, the output of the accelerometer or gyroscope may be used by computing device 1600 to render immersive augmented reality or virtual reality.

[0123] Power supply 1616 may include hard-wired power supply, battery power supply, or a combination thereof. Power supply 1616 may supply power to computing device 1600 to allow the components of computing device 1600 to operate.

[0124] The presentation component 1618 may include a display (such as a monitor, touch screen, television screen, head-up display (HUD), other display types, or combinations thereof), speakers, and / or other presentation components. The presentation component 1618 may receive data from other components (such as GPU 1608, CPU 1606, DPU, etc.) and output that data (such as as images, videos, sounds, etc.).

[0125] Example Data Center

[0126] Figure 17 An example data center 1700 that may be used in at least one embodiment of this disclosure is shown. The data center 1700 may include a data center infrastructure layer 1710, a framework layer 1720, a software layer 1730, and / or an application layer 1740.

[0127] like Figure 17 As shown, the data center infrastructure layer 1710 may include a resource coordinator 1712, grouped computing resources 1714, and node computing resources (“nodes CR”) 1716(1)-1716(N), where “N” represents any complete positive integer. In at least one embodiment, nodes CR 1716(1)-1716(N) may include, but are not limited to, any number of central processing units (“CPUs”) or other processors (including DPUs, accelerators, field-programmable gate arrays (FPGAs), graphics processors or graphics processing units (GPUs), etc.), memory devices (e.g., dynamic read-only memory), storage devices (e.g., solid-state or disk drives), network input / output (“NW I / O”) devices, network switches, virtual machines (“VMs”), power modules and / or cooling modules, etc. In some embodiments, one or more node CRs from nodes CR 1716(1)-1716(N) may correspond to servers having one or more of the aforementioned computing resources. In addition, in some embodiments, nodes CR1716(1)-17161(N) may include one or more virtual components, such as vGPU, vCPU, etc., and / or one or more nodes CRs1916(1)-1916(N) may correspond to virtual machines (VMs).

[0128] In at least one embodiment, the grouped computing resources 1714 may include individual groups of nodes CR1716 housed within one or more racks (not shown), or multiple racks housed within a data center at different geographical locations (also not shown). Individual groups of nodes CR1716 within the grouped computing resources 1714 may include grouped computing, networking, memory, or storage resources that can be configured or allocated to support one or more workloads. In at least one embodiment, several nodes CR1716, including CPUs, GPUs, DPUs, and / or other processors, may be grouped within one or more racks to provide computing resources to support one or more workloads. One or more racks may also include any number of power modules, cooling modules, and / or network switches in any combination.

[0129] Resource coordinator 1712 may be configured or otherwise control one or more nodes CR1716(1)-1716(N) and / or grouped computing resources 1714. In at least one embodiment, resource coordinator 1712 may include a Software Design Infrastructure (“SDI”) management entity for data center 1700. Resource coordinator 1712 may include hardware, software, or some combination thereof.

[0130] In at least one embodiment, such as Figure 17As shown, framework layer 1720 may include job scheduler 1728, configuration manager 1734, resource manager 1736, and / or distributed file system 1738. Framework layer 1720 may include a framework of software 1732 supporting software layer 1730 and / or one or more applications 1742 supporting application layer 1740. Software 1732 or application 1742 may respectively contain web-based service software or applications, such as those provided by Amazon Web Services, Google Cloud, and Microsoft Azure. Framework layer 1720 may be, but is not limited to, a free and open-source software web application framework (such as Apache Spark™ (hereinafter “Spark”)) that can perform large-scale data processing (e.g., “big data”) using distributed file system 1738. In at least one embodiment, job scheduler 1728 may include Spark drivers to facilitate the scheduling of workloads supported by different layers of data center 1700. Configuration manager 1734 may be able to configure different layers, such as software layer 1730 and framework layer 1720 (which includes Spark and distributed file system 1738 for supporting large-scale data processing). Resource manager 1736 may be able to manage computing resources mapped to or allocated to clusters of distributed file system 1778 and job scheduler 1728 to support distributed file system 1738 and job scheduler 1728. In at least one embodiment, clustered or grouped computing resources may include grouped computing resources 1714 in data center infrastructure layer 1710. Resource manager 1736 may coordinate with resource coordinator 1712 to manage these mapped or allocated computing resources.

[0131] In at least one embodiment, the software 1732 included in software layer 1730 may include software used in at least a portion of nodes CR1716(1)-1716(N), grouped computing resources 1714, and / or the distributed file system 1738 of framework layer 1720. One or more types of software may include, but are not limited to, internet web search software, email virus scanning software, database software, and streaming video content software.

[0132] In at least one embodiment, the application 1742 included in the application layer 1740 may include one or more types of applications used at least in part by nodes CR1716(1)-1716(N), grouped computing resources 1714, and / or the distributed file system 1738 of the framework layer 1720. One or more types of applications may include, but are not limited to, any number of genomics applications, cognitive computing, and machine learning applications, including training or inference software, machine learning framework software (e.g., PyTorch, TensorFlow, Caffe, etc.), and / or other machine learning applications used in combination with one or more embodiments.

[0133] In at least one embodiment, any of the configuration manager 1734, resource manager 1736, and resource coordinator 1712 can implement any number and type of self-modification actions based on any amount and type of data obtained in any technically feasible manner. Self-modification actions can free the data center operator of data center 1700 from making potentially poor configuration decisions and potentially avoid underutilization and / or poor performance of the data center.

[0134] According to one or more embodiments described herein, data center 1700 may include tools, services, software, or other resources to train one or more machine learning models or to use one or more machine learning models to predict or infer information. For example, one or more machine learning models may be trained by using the software and / or computing resources described above with respect to data center 1700 to compute weight parameters according to a neural network architecture. In at least one embodiment, a trained or deployed machine learning model corresponding to one or more neural networks may be used to infer or predict information using the resources described above with respect to data center 1700 by using weight parameters computed through one or more training techniques, such as, but not limited to, those described herein.

[0135] In at least one embodiment, the data center 1700 may use a CPU, application-specific integrated circuit (ASIC), GPU, FPGA, and / or other hardware (or corresponding virtual computing resources) to perform training and / or inference using the aforementioned resources. Furthermore, one or more of the software and / or hardware resources described above may be configured to allow a user to train or perform services that infer information, such as image recognition, speech recognition, or other artificial intelligence services.

[0136] Example network environment

[0137] A network environment suitable for implementing embodiments of this disclosure may include one or more client devices, servers, network-attached storage (NAS), other backend devices, and / or other device types. Client devices, servers, and / or other device types (e.g., each device) may be... Figure 16 This is implemented on one or more instances of computing devices 1600—for example, each device may include similar components, features, and / or functions of one or more computing devices 1600. Furthermore, in the case of implementing backend devices (e.g., servers, NAS, etc.), the backend devices may be included as part of a data center 1700, examples of which are described in this document. Figure 17 To describe in more detail.

[0138] Components of a network environment can communicate with each other via a network, which can be wired, wireless, or both. A network can include multiple networks or one of multiple networks. For example, a network can include one or more wide area networks (WANs), one or more local area networks (LANs), one or more public networks (such as the Internet and / or the Public Switched Telephone Network (PSTN)), and / or one or more private networks. Where the network includes a wireless telecommunications network, components such as base stations, communication towers, or even access points (and other components) can provide wireless connectivity.

[0139] A compatible network environment may include one or more peer-to-peer network environments (in which case the server may not be included in the network environment) and one or more client-server network environments (in which case one or more servers may be included in the network environment). In a peer-to-peer network environment, the functionality described herein for the server can be implemented on any number of client devices.

[0140] In at least one embodiment, the network environment may include one or more cloud-based network environments, distributed computing environments, combinations thereof, etc. The cloud-based network environment may include a framework layer, a job scheduler, a resource manager, and a distributed file system implemented on one or more servers, which may include one or more core network servers and / or edge servers. The framework layer may include a framework supporting software at the software layer and / or application at the application layer. The software or application may respectively include network-based service software or applications. In embodiments, one or more client devices may use the network-based service software or applications (e.g., by accessing the service software and / or applications via one or more application programming interfaces (APIs)). The framework layer may be, but is not limited to, a free and open-source software network application framework that can use a distributed file system for large-scale data processing (e.g., "big data").

[0141] A cloud-based network environment can provide cloud computing and / or cloud storage for any combination of the computing and / or data storage functions (or one or more portions thereof) described herein. Any of these different functions can be distributed across multiple locations from a central or core server (e.g., distributed across one or more data centers at the state, region, country, global, etc.). The core server may assign at least a portion of the functionality to the edge server if the connection to the user (e.g., a client device) is relatively close to the edge server. A cloud-based network environment can be private (e.g., limited to a single organization), public (e.g., available to many organizations), and / or a combination thereof (e.g., a hybrid cloud environment).

[0142] One or more client devices may include the information described in this article. Figure 16 At least some of the components, features, and functions of one or more example computing devices 1600 described. By way of example and not limitation, the client device may be implemented as a personal computer (PC), laptop computer, mobile device, smartphone, tablet computer, smartwatch, wearable computer, personal digital assistant (PDA), MP3 player, virtual reality headset, global positioning system (GPS) or device, video player, camera, surveillance equipment or system, vehicle, ship, spacecraft, virtual machine, drone, robot, handheld communication device, hospital equipment, gaming equipment or system, entertainment system, vehicle computer system, embedded system controller, remote control, electrical appliance, consumer electronics device, workstation, edge device, any combination of these depicted devices, or any other suitable device.

[0143] This disclosure can be described in the general context of machine-usable instructions or computer code, including computer-executable instructions such as program modules, which are executed by a computer or other machine such as a personal digital assistant or other handheld device. Typically, a program module, including routines, programs, objects, components, data structures, etc., refers to code that performs a specific task or implements a specific abstract data type. This disclosure can be practiced in a wide variety of system configurations, including handheld devices, consumer electronics, general-purpose computers, more specialized computing devices, etc. This disclosure can also be practiced in distributed computing environments where tasks are performed by remote processing devices linked via a communication network.

[0144] As used herein, the phrase “and / or” relating to two or more elements should be interpreted as referring to only one element or a combination of elements. For example, “element A, element B, and / or element C” can include only element A, only element B, only element C, element A and element B, element A and element C, element B and element C, or element A, B, and C. Additionally, “at least one of element A or element B” can include at least one of element A, at least one of element B, or at least one of element A and at least one of element B. Furthermore, “at least one of element A and element B” can include at least one of element A, at least one of element B, or at least one of element A and at least one of element B.

[0145] The subject matter of this disclosure has been specifically described herein to satisfy legal requirements. However, the description itself is not intended to limit the scope of this disclosure. Rather, the inventors have considered that the claimed subject matter may also be embodied in other ways to include different steps or combinations of steps similar to those described herein, combined with other current or future techniques. Furthermore, although the terms “step” and / or “block” may be used herein to denote different elements of the method employed, these terms should not be construed as implying any particular order between the various steps disclosed herein, unless the order of the various steps is explicitly described.

[0146] Example paragraph

[0147] A: A method comprising: obtaining data representing one or more modifications that have occurred with respect to a first version of a model for generating a second version of the model; generating a checksum for the second version of the model based at least on the data representing the one or more modifications; and providing the second version of the model and the checksum to one or more computing devices, wherein the second version of the model is verifiable using the checksum.

[0148] B: As described in paragraph A, wherein the data representing the one or more modifications represents at least one of the following: the architecture associated with the second version of the model; the dataset used to train the second version of the model; the risk score associated with the second version of the model; the license associated with the second version of the model; the performance score associated with the second version of the model; or the second version of the model for one or more hardware types for which it is optimized.

[0149] C: The method as described in paragraph A or B, wherein generating the checksum of the second version of the model comprises: generating information describing the one or more modifications based at least on the data representing the one or more modifications; and determining the checksum associated with the second version of the model based at least on the information.

[0150] D: The method as described in paragraph C, wherein determining the checksum associated with the second version of the model based at least on the information comprises: encoding the information to generate encoded information; and applying one or more algorithms to the encoded information to obtain the checksum associated with the second version of the model.

[0151] E: The method as described in paragraph C, wherein the information describing the one or more modifications includes at least one of the following: a first text describing the one or more modifications; a second text describing the dataset used to perform the one or more modifications; a third text describing the performance changes associated with the second version of the model; a fourth text describing the time when the one or more modifications occurred; a fifth text describing the one or more users performing the one or more modifications; or a sixth text describing the hardware performing the one or more modifications.

[0152] F: The method as described in any of paragraphs AE further includes: determining information describing the one or more modifications based at least on the checksum; and providing the information to the one or more computing devices.

[0153] G: The method as described in any of paragraphs AF further comprises: providing the first version of the model to one or more second computing devices, wherein obtaining the data representing the one or more modifications is based at least on modifying the first version of the model using the one or more second computing devices.

[0154] H: The method as described in any of paragraphs AG further comprises: obtaining second data representing one or more second modifications that occurred with respect to a first version of the model or a second version of the model for generating a third version of the model; generating a second checksum associated with the third version of the model based at least on the second data representing the one or more second modifications; and storing the second checksum in association with the third version of the model.

[0155] I: The method as described in any of paragraphs AH, wherein the one or more modifications are associated with at least one of: fine-tuning the first version of the model to generate a second version of the model; optimizing the first version of the model for one or more hardware components; or performing quantization on the first version of the model to generate a second version of the model.

[0156] J: A system comprising: one or more processors configured to: provide a first version of a model to one or more computing devices; obtain data representing one or more modifications that occurred to generate a second version of the model, based at least on modifying the first version of the model using the one or more computing devices; generate a unique identifier for the second version of the model, based at least on the data; and store the unique identifier in association with the second version of the model.

[0157] K: The system as described in paragraph J, wherein the one or more processors are further configured to: provide the unique identifier and the second version of the model to one or more second computing devices, wherein the second version of the model can be verified using the unique identifier.

[0158] L: The system as described in paragraph J or K, wherein the one or more processors are further configured to: generate information describing the one or more modifications, at least based on the data, wherein the unique identifier is generated at least based on the information.

[0159] M: The system as described in paragraph L, wherein the unique identifier for generating the second version of the model comprises: encoding the information to generate encoded information; and generating the unique identifier associated with the second version of the model based at least on hashing the encoded information using one or more algorithms.

[0160] N: A system as described in any of paragraphs JM, wherein the one or more processors are further configured to: determine information describing the one or more modifications based at least on the unique identifier; and provide the information and a second version of the model to the one or more second computing devices.

[0161] O: A system as described in any of paragraphs JN, wherein the one or more processors are further configured to: provide a second version of the model to one or more second computing devices; obtain data representing one or more second modifications that occurred to generate a third version of the model, based at least on modifying the second version of the model using the one or more second computing devices; generate a second unique identifier associated with the third version of the model, based at least on the data; and store the second unique identifier in association with the third version of the model.

[0162] P: The system as described in paragraph O, wherein the one or more processors are further configured to: provide first information describing the one or more modifications to one or more computing devices and using the unique identifier; provide second information describing the one or more second modifications to one or more computing devices and using the second unique identifier; receive a request for a second version of the model from the one or more computing devices; and provide the second version of the model to the one or more computing devices and at least based on the request.

[0163] Q: A system as described in any of paragraphs JP, wherein the system includes at least one of the following: a control system for an autonomous or semi-autonomous machine; a perception system for an autonomous or semi-autonomous machine; a system for performing one or more analog operations; a system for performing one or more digital twin operations; a system for performing optical transmission simulation; a system for performing collaborative content creation for 3D assets; a system for providing one or more cloud gaming applications; a system for performing one or more deep learning operations; a system implemented using an edge device; a system implemented using a robot; a system for performing one or more generative AI operations; a system for performing one or more operations using one or more small language models; a system for performing one or more operations using one or more large language models. Systems for performing operations using LLM models; systems for performing operations using one or more Visual Language Models (VLMs); systems for performing operations using one or more Multimodal Language Models (MLMs); systems for performing one or more conversational AI operations; systems for generating synthetic data; systems for presenting at least one of virtual reality content, augmented reality content, or mixed reality content; systems for implementing one or more MLMs; systems for using or deploying one or more inference microservices; systems for deploying one or more machine learning models contained in services or microservices and OS-level virtualization packages (e.g., containers); systems containing one or more virtual machines (VMs); systems implemented at least partially in a data center; or systems implemented at least partially using cloud computing resources.

[0164] R: One or more processors, the one or more processors comprising: processing circuitry for: generating information describing one or more modifications made to a first version of the model for generating a second version of the model; generating a unique identifier for the second version of the model based at least on the information; and storing the unique identifier in association with the second version of the model.

[0165] S: One or more processors as described in paragraph R, wherein the unique identifier is generated by: encoding the information to generate encoded information; and applying one or more algorithms to the encoded information to obtain the unique identifier of the second version of the model.

[0166] T: One or more processors as described in paragraph R or S, wherein the one or more processors are included in at least one of the following: a control system for an autonomous or semi-autonomous machine; a perception system for an autonomous or semi-autonomous machine; a system for performing one or more analog operations; a system for performing one or more digital twin operations; a system for performing optical transmission simulation; a system for performing collaborative content creation for 3D assets; a system for providing one or more cloud gaming applications; a system for performing one or more deep learning operations; a system implemented using edge devices; a system implemented using robots; a system for performing one or more generative AI operations; a system for performing one or more operations using one or more small language models; a system for performing one or more operations using one or more... Systems that perform operations using multiple large language model LLMs; systems that perform operations using one or more visual language model VLMs; systems that perform operations using one or more multimodal language models; systems that perform one or more conversational AI operations; systems that generate synthetic data; systems that render at least one of virtual reality content, augmented reality content, or mixed reality content; systems that implement one or more multimodal language models; systems that use or deploy one or more inference microservices; systems that deploy one or more machine learning models within services or microservices and OS-level virtualization packages (e.g., containers); systems that contain one or more virtual machines VMs; systems that are at least partially implemented in a data center; or systems that are at least partially implemented using cloud computing resources.

[0167] U: A method comprising: determining one or more modifications made to a first version of a model for generating a second version of the model; obtaining a model card associated with the first version of the model; updating the model card to include at least a portion of information relating to the one or more modifications to generate an updated model card; determining to provide the second version of the model to the computing device based at least on the updated model card and one or more capabilities associated with a computing device; and transmitting data to the computing device for executing the second version of the model.

[0168] V: The method described in paragraph U further includes: obtaining a unique identifier associated with the second version of the model, wherein the determination of the one or more modifications that occurred to the first version of the model is based at least on the unique identifier.

[0169] W: The method as described in paragraphs U or V, wherein the information relating to the one or more modifications is determined by: obtaining a unique identifier associated with the second version of the model; and processing the unique identifier to determine the information relating to the one or more modifications.

[0170] X: The method as described in any of paragraphs UW, wherein the information relating to the one or more modifications is determined by: performing one or more performance tests associated with the second version of the model; and identifying the information relating to the one or more modifications based at least on the one or more performance tests.

[0171] Y: The method as described in any of paragraphs UX, wherein the information relating to the one or more modifications includes at least one of the following: first information indicating the type of modification that occurred to the first version of the model; second information indicating the time when the one or more modifications occurred; third information indicating one or more hardware components that the second version of the model is optimized for; fourth information indicating one or more performance characteristics associated with executing the second version of the model using the one or more hardware components; fifth information indicating one or more performance characteristics associated with the second version of the model; or a unique identifier associated with the second version of the model.

[0172] Z: The method as described in any of paragraphs UY further includes: obtaining second information relating to one or more second modifications made to the second version of the model for generating a third version of the model; and updating the updated model card to include at least a portion of the second information.

[0173] AA: The method as described in any of paragraphs UZ further comprises at least one of the following: storing the updated model card in association with the second version of the model; or providing at least one of the second version of the model or the updated model card to one or more systems.

[0174] AB: A system comprising: one or more processors configured to: determine information relating to one or more modifications made to a first version of a model for generating a second version of the model; access a model card associated with the first version of the model; update the model card to generate an updated model card including at least a portion of the information; and store the updated model card in association with the second version of the model.

[0175] AC: The system as described in paragraphs AB, wherein the one or more processors are further configured to: determine, at least based on the updated model card and one or more capabilities associated with the computing device, to provide the second version of the model to the computing device; and to send data to the computing device for executing the second version of the model.

[0176] AD: The system as described in paragraphs AB or AC, wherein the one or more processors are further configured to: obtain a unique identifier associated with the second version of the model; and determine, at least based on the unique identifier, the one or more modifications that have occurred to the first version of the model, wherein the model card is updated at least based on the determination of the one or more modifications that have occurred.

[0177] AE: A system as described in any of paragraphs AB-AD, wherein the information relating to the one or more modifications is determined by: obtaining a unique identifier associated with the second version of the model; and processing the unique identifier to determine the information relating to the one or more modifications.

[0178] AF: A system as described in any of paragraphs AB-AE, wherein the information relating to the one or more modifications is determined by: performing one or more performance tests associated with the second version of the model; and determining the information relating to the one or more modifications based at least on the one or more performance tests.

[0179] AG: A system as described in any of paragraphs AB-AF, wherein the information relating to the one or more modifications is determined by: obtaining data associated with performing the one or more modifications to generate a second version of the model; and determining the information relating to the one or more modifications based at least on the data.

[0180] AH: A system as described in any of paragraphs AB-AG, wherein the information relating to the one or more modifications includes at least one of the following: first information indicating the type of modification that occurred to the first version of the model; second information indicating the time when the one or more modifications occurred; third information indicating one or more hardware components that the second version of the model is optimized for; fourth information indicating one or more performance characteristics associated with executing the second version of the model using the one or more hardware components; fifth information indicating one or more performance characteristics associated with the second version of the model; or a unique identifier associated with the second version of the model.

[0181] AI: A system as described in any of paragraphs AB-AH, wherein the one or more processors are further configured to: determine second information relating to one or more second modifications made to the first version of the model for generating a third version of the model; and update the updated model card to include at least a portion of the second information.

[0182] AJ: A system as described in any of paragraphs AI, wherein: the one or more modifications are associated with a second version of the model optimized using a first type of hardware; the at least part of the information indicates one or more first performance characteristics associated with executing the second version of the model on the first type of hardware; the one or more second modifications are associated with a third version of the model optimized using a second type of hardware; and the at least part of the second information indicates one or more second performance characteristics associated with executing the second version of the model on the second type of hardware.

[0183] AK: A system as described in any one of paragraphs AB-AJ, wherein the system includes at least one of: a control system for an autonomous or semi-autonomous machine; a perception system for an autonomous or semi-autonomous machine; a system for performing one or more analog operations; a system for performing one or more digital twin operations; a system for performing optical transmission simulation; a system for performing collaborative content creation for 3D assets; a system for providing one or more cloud gaming applications; a system for performing one or more deep learning operations; a system implemented using edge devices; a system implemented using robots; a system for performing one or more generative AI operations; a system for performing one or more operations using one or more small language models (SLMs); a system for performing one or more operations using one or more... Systems for performing operations using large language model (LLM); systems for performing operations using one or more visual language models (VLM); systems for performing operations using one or more multimodal language models; systems for performing one or more conversational AI operations; systems for generating synthetic data; systems for presenting at least one of virtual reality content, augmented reality content, or mixed reality content; systems for implementing one or more multimodal language models; systems for using or deploying one or more inference microservices; systems for deploying one or more machine learning models contained in services or microservices and OS-level virtualization packages (e.g., containers); systems containing one or more virtual machines (VMs); systems implemented at least partially in a data center; or systems implemented at least partially using cloud computing resources.

[0184] AL: One or more processors, the one or more processors comprising: processing circuitry for: obtaining a unique identifier associated with a version of a model; determining, at least based on the unique identifier, information relating to one or more modifications to a previous version of the model for generating the version of the model; updating, at least partially, a model card associated with the previous version of the model using the information to generate an updated model card; and storing the updated model card in association with the version of the model.

[0185] AM: One or more processors as described in paragraph AL, wherein the processing circuitry is further configured to: determine, based at least on the updated model card and one or more capabilities associated with the computing device, provide the version of the model to the computing device; and send data to the computing device for executing the second version of the model.

[0186] AN: One or more processors as described in paragraphs AL or AM, wherein the one or more processors are included in at least one of the following: a control system for an autonomous or semi-autonomous machine; a perception system for an autonomous or semi-autonomous machine; a system for performing one or more analog operations; a system for performing one or more digital twin operations; a system for performing optical transmission simulation; a system for performing collaborative content creation for 3D assets; a system for providing one or more cloud gaming applications; a system for performing one or more deep learning operations; a system implemented using edge devices; a system implemented using robots; a system for performing one or more generative AI operations; a system for performing one or more operations using one or more small language models (SLMs); a system for performing one or more operations using a... Systems that perform operations using one or more large language model (LLM); systems that perform operations using one or more visual language model (VLM); systems that perform operations using one or more multimodal language models; systems that perform one or more conversational AI operations; systems that generate synthetic data; systems that render at least one of virtual reality content, augmented reality content, or mixed reality content; systems that implement one or more multimodal language models; systems that use or deploy one or more inference microservices; systems that deploy one or more machine learning models within services or microservices and OS-level virtualization packages (e.g., containers); systems that contain one or more virtual machines (VMs); systems that are at least partially implemented in a data center; or systems that are at least partially implemented using cloud computing resources.

Claims

1. A method, the method comprising: Obtain data representing one or more modifications that occurred with respect to the first version of the model, used to generate the second version of the model; A checksum for the second version of the model is generated based at least on the data representing the one or more modifications; as well as The second version of the model and the checksum are provided to one or more computing devices, wherein the second version of the model can be verified using the checksum.

2. The method as described in claim 1, wherein, The data representing the one or more modifications represents at least one of the following: The architecture associated with the second version of the model; The second version of the dataset used to train the model; The risk score associated with the second version of the model; The license associated with the second version of the model; The performance score associated with the second version of the model; or The second version of the model is optimized for one or more hardware types.

3. The method as described in claim 1, wherein, The checksum used to generate the second version of the model includes: At least based on the data representing the one or more modifications, information describing the one or more modifications is generated; and The checksum associated with the second version of the model is determined based at least on the information provided.

4. The method of claim 3, wherein, Determining the checksum associated with the second version of the model based at least on the information includes: Encode the information to generate encoded information; and One or more algorithms are applied to the encoded information to obtain the checksum associated with the second version of the model.

5. The method of claim 3, wherein, The information describing the one or more modifications includes at least one of the following: The first text describing the one or more modifications; The second text describing the dataset used to perform the one or more modifications; A third text describing the performance changes associated with the second version of the model; A fourth text describing the time when the one or more modifications occurred; The fifth text describes the one or more users who performed the one or more modifications; or The sixth text describes the hardware that performs the one or more modifications.

6. The method of claim 1, further comprising: Based at least on the checksum, information describing the one or more modifications is determined; as well as The information is provided to the one or more computing devices.

7. The method of claim 1, further comprising: The first version of the model is provided to one or more second computing devices. Wherein, obtaining the data representing the one or more modifications is based at least on the first version of the model modified using the one or more second computing devices.

8. The method of claim 1, further comprising: Obtain second data representing one or more second modifications that occurred with respect to either the first version of the model or the second version of the model, for generating a third version of the model; A second checksum associated with the third version of the model is generated, based at least on the second data representing the one or more second modifications; as well as The second checksum is stored in association with the third version of the model.

9. The method of claim 1, wherein, The one or more modifications are associated with at least one of the following: Fine-tuning the first version of the model to generate the second version of the model; The first version of the model is optimized for one or more hardware components; or Quantization is performed on the first version of the model to generate the second version of the model.

10. A system comprising: One or more processors are used for: Provide the first version of the model to one or more computing devices; At least based on modifying the first version of the model using the one or more computing devices, data representing one or more modifications that occurred to generate the second version of the model are obtained; Based at least on the data, generate a unique identifier for the second version of the model; as well as The unique identifier is stored in association with the second version of the model.

11. The system of claim 10, wherein, The one or more processors are also used for: The unique identifier and the second version of the model are provided to one or more second computing devices. The second version of the model can be verified using the unique identifier.

12. The system of claim 10, wherein, The one or more processors are also used for: Based at least on the data, generate information describing the one or more modifications. The unique identifier is generated at least based on the information.

13. The system of claim 12, wherein, The unique identifier used to generate the second version of the model includes: Encode the information to generate encoded information; and The unique identifier associated with the second version of the model is generated by hashing the encoded information using at least one or more algorithms.

14. The system of claim 10, wherein, The one or more processors are also used for: Based at least on the unique identifier, determine information describing the one or more modifications; and The information and the second version of the model are provided to the one or more second computing devices.

15. The system of claim 10, wherein, The one or more processors are also used for: The second version of the model is provided to one or more second computing devices; At least based on modifying the second version of the model using the one or more second computing devices, data representing one or more second modifications that occurred to generate the third version of the model are obtained; Based at least on the data, generate a second unique identifier associated with the third version of the model; as well as The second unique identifier is stored in association with the third version of the model.

16. The system of claim 15, wherein, The one or more processors are also used for: Provide first information describing the one or more modifications to one or more computing devices using the unique identifier; Provide second information describing the one or more second modifications to the one or more computing devices and using the second unique identifier; Receive a request for the second version of the model from one or more computing devices; as well as The second version of the model is provided to the one or more computing devices and at least based on the request.

17. The system of claim 10, wherein, The system is included in at least one of the following: Control systems for autonomous or semi-autonomous machines; Sensing systems for autonomous or semi-autonomous machines; A system for performing one or more simulation operations; A system for performing one or more digital twin operations; A system for performing optical transmission simulation; A system for performing collaborative content creation for 3D assets; A system that provides one or more cloud gaming applications; A system for performing one or more deep learning operations; Systems implemented using edge devices; Systems implemented using robots; A system for performing one or more generative AI operations; A system for performing one or more operations using one or more small language models; A system for performing operations using one or more large language model LLMs; A system for performing operations using one or more Visual Language Models (VLMs); A system for performing operations using one or more multimodal language models; A system for performing one or more conversational AI operations; A system for generating synthetic data; A system for presenting at least one of virtual reality content, augmented reality content, or mixed reality content; A system that implements one or more multimodal language models; A system that uses or deploys one or more inference microservices; A system that includes one or more machine learning models deployed in services or microservices and OS-level virtualization packages (e.g., containers); A system containing one or more virtual machines (VMs); A system that is at least partially implemented in a data center; or A system that utilizes cloud computing resources at least in part.

18. One or more processors, said one or more processors comprising: Processing circuitry, used for: Generate information describing one or more modifications made to the first version of the model to generate a second version of the model; Based at least on the information, generate a unique identifier for the second version of the model; as well as The unique identifier is stored in association with the second version of the model.

19. One or more processors as claimed in claim 18, wherein, The unique identifier is generated from the following: Encode the information to generate encoded information; as well as One or more algorithms are applied to the encoded information to obtain the unique identifier of the second version of the model.

20. The processor of claim 18, wherein the processor is included in at least one of the following: Control systems for autonomous or semi-autonomous machines; Sensing systems for autonomous or semi-autonomous machines; A system used to perform one or more simulation operations; A system for performing one or more digital twin operations; A system for performing optical transmission simulation; A system for performing collaborative content creation for 3D assets; A system that provides one or more cloud gaming applications; A system for performing one or more deep learning operations; Systems implemented using edge devices; Systems implemented using robots; A system for performing one or more generative AI operations; A system for performing one or more operations using one or more small language models; A system for performing operations using one or more large language model LLMs; A system for performing operations using one or more Visual Language Models (VLMs); A system for performing operations using one or more multimodal language models; A system for performing one or more conversational AI operations; A system for generating synthetic data; A system for presenting at least one of virtual reality content, augmented reality content, or mixed reality content; A system that implements one or more multimodal language models; A system that uses or deploys one or more inference microservices; A system that includes one or more machine learning models deployed in services or microservices and OS-level virtualization packages (e.g., containers); A system containing one or more virtual machines (VMs); A system that is at least partially implemented in a data center; or A system that utilizes cloud computing resources at least in part.

Images