DYNAMIC UPDATES OF MACHINE LEARNING MODEL CARDS BASED ON MODEL CHANGES

Abstract

This document describes, through various examples, the process of updating model maps with information associated with modifications to machine learning models. For instance, the systems and methods described herein can automatically update model maps with additional information about modifications made to models. To update a model map, the unique identifier can be used to identify that a modification has been made to a model—such as through fine-tuning, optimization, quantization, and / or another modification technique—to generate a new version of the model. Additional information about the modification is then gathered, such as by decoding the unique identifier, testing the new version of the model, and / or using data that describes how the model was modified.The model map can then be automatically updated with the additional information. Furthermore, this process can be repeated if additional modifications are made to the model to generate new versions of the model.

Description

BACKGROUND

[0001] Online repositories allow users to download, modify (e.g., update, optimize, etc.), and share models (e.g., machine learning models, neural networks, classifiers, etc.). For example, a user can use an online repository to modify an initial version of a model—perhaps by performing fine-tuning, optimization, and / or quantization—to generate a second version of the model. The user can then share this second version with other users using the online repository. However, traditional systems that provide online repositories for models may lack sufficient mechanisms for tracking and / or auditing modifications made to the models.For example, if a user creates a new version of a model using fine-tuning, other users may not be able to identify which dataset was used to perform the fine-tuning, the accuracy of the model after fine-tuning, and / or that the new version of the model was created by performing fine-tuning. This can make it difficult for traditional systems to track the integrity and / or origin of the models.

[0002] To address these shortcomings, some conventional systems store model maps containing information about available models. A model map might contain various pieces of information about a particular model—such as the model's development process, training data, performance metrics, potential biases, limitations, intended use cases, and / or out-of-scope applications—enabling end users to make informed decisions about deploying and / or using the model. However, when models are modified to create new versions, these model maps may not be updated with the information describing the modifications.Even when model cards are updated, users of traditional systems must manually identify the information associated with the modifications and enter this information into the model cards. Traditional systems are therefore prone to errors, such as when the user enters incorrect information into the model cards and / or fails to identify relevant information describing the modifications. SUMMARY

[0003] The invention is defined in the claims. For the purpose of illustrating the invention, aspects and embodiments are described herein that may or may not fall within the scope of protection of the claims.

[0004] Embodiments of the present disclosure relate to the identifier-based tracking and validation of machine learning model modifications. For example, the systems and methods described herein can generate unique identifiers—such as checksums, hashes, and / or the like—that track modifications to models. For instance, if an initial version of a model is modified (e.g., updated, changed, optimized, extended, etc.) to generate a second version of the model—such as through fine-tuning, optimization, quantization, and / or other modification techniques—data can be obtained that represents the modifications. This data can then be used to generate information (e.g., text, messages, notes, comments, etc.) that describes the modifications. Additionally, the information can be encoded using one or more encoders and / or processed using one or more algorithms (e.g., algorithms, algorithms, etc.).The data is processed (e.g., using one or more hash algorithms) to generate a unique identifier for the second version of the model. Subsequently, systems and procedures are described that use these unique identifiers to perform various tasks, such as verifying the integrity of the different versions of the models, providing information about modifications, and / or similar functions.

[0005] In contrast to conventional systems, the systems of this disclosure, in some embodiments, generate unique identifiers that can be used to track modifications made to models, verify the integrity of the models, and / or identify specific versions of models for further modification and / or use. For example, if a user attempts to retrieve a model from an online repository, they can use the unique identifiers to determine information about how the versions of the model were modified, performance ratings for the versions of the model, the date on which the versions of the model were created, the identifiers of the persons who created the versions of the model, and / or other change information.Thus, the user can use the unique identifiers to select a version of the model to retrieve and / or to verify the integrity of the model after retrieval. In other words, the systems of this disclosure can use the unique identifiers to guarantee the integrity and / or origin of models provided using the online repository.

[0006] Embodiments of the present disclosure further relate to dynamic updates of machine learning model maps based on model changes. For example, the systems and methods described herein can automatically update model maps with additional information about modifications made to models. To update a model map, the unique identifier can be used to identify that a modification has been made to a model—such as through fine-tuning, optimization, quantization, and / or another modification technique—to generate a new version of the model. Additional information about the modification is then obtained, such as by decoding the unique identifier, testing the new version of the model, and / or using data describing how the model was modified.The model map associated with the model can then be automatically updated with the additional information. Furthermore, this process can be repeated if additional modifications are made to the model to generate new versions of the model.

[0007] In contrast to conventional systems, the systems of the present disclosure, in some embodiments, automatically determine that modifications have been made to models, obtain additional information associated with the modifications, and then update the model maps for the models using this additional information. As described above, conventional systems may, for example, store model maps containing information about the early versions of a model. However, the model maps stored by conventional systems are generally not updated when models are modified and / or require user input for the update, which is error-prone. The systems of the present disclosure provide improvements in that the model maps are continuously updated to include additional information about newer versions of models.Furthermore, by obtaining the additional information using the techniques described herein, the systems of this disclosure ensure that the model maps are accurately updated to reflect the actual properties and / or performance information about the models.

[0008] Further features of the disclosure are characterized by the independent and dependent claims.

[0009] Any feature in one aspect of the disclosure can be applied in any suitable combination to other aspects of the disclosure. In particular, procedural aspects can be applied to apparatus or system aspects, and vice versa.

[0010] Furthermore, features implemented in hardware can be implemented in software and vice versa. Any reference to software and hardware features herein should be interpreted accordingly.

[0011] Each system or device feature described herein can also be provided as a process feature, and vice versa. System and / or device aspects that are functionally described (including means plus functional features) can alternatively be expressed in terms of their corresponding structure, such as a suitably programmed processor and allocated working memory.

[0012] It is also understood that certain combinations of the various features described and defined in each aspect of the revelation can be implemented and / or provided and / or used independently of one another.

[0013] The disclosure also provides computer programs and computer program products comprising software code designed to perform one of the methods described herein when executed on a data processing device and / or to embody one of the device and system features described herein, including one or all component steps of a method.

[0014] The disclosure also provides a computer or computing system (including networked or distributed systems) with an operating system that supports a computer program for carrying out one of the methods described herein and / or for embodying one of the device or system features described herein.

[0015] The disclosure also provides a computer-readable medium on which one or more of the aforementioned computer programs are stored.

[0016] The revelation also provides a signal that carries one or more of the aforementioned computer programs.

[0017] The disclosure extends to methods and / or devices and / or systems as described herein with reference to the accompanying drawings.

[0018] Aspects and embodiments of the disclosure will now be described purely by way of example with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The present systems and procedures for tracking and verifying modifications to machine learning models are described in detail below with reference to the accompanying drawings, wherein: Fig. 1 An example of a process for using unique identifiers for tracking and verifying modifications to models according to some embodiments of the present disclosure is illustrated; Fig. 2 An example of the storage of different versions of a model that have been modified, according to some embodiments of the present disclosure, is illustrated; Fig. 3 An example of obtaining data representing features and / or modifications associated with a model is illustrated according to some embodiments of the present disclosure; Fig. 4 An example of the generation of structured information associated with a version of a model is illustrated according to some embodiments of the present disclosure; Fig. 5 An example of the generation of unique identifiers that are assigned to different versions of a model is illustrated according to some embodiments of the present disclosure; Fig. 6 An example of the provision of structured information associated with different versions of a model is illustrated according to some embodiments of the present disclosure; Fig. 7 An example of a process for automatically updating model maps to include additional information for modifications to models is illustrated according to some embodiments of the present disclosure; Fig. 8 An example of decoding a unique identifier to obtain additional information associated with a modification of a model is illustrated according to some embodiments of the present disclosure; Fig. 9 illustrates an example of the testing of a model for determining performance data according to some embodiments of the present disclosure; Fig. Figures 10A-10C illustrate an example of updating a model map to include additional information associated with modifications to a model, according to some embodiments of the present disclosure; Fig. 11 a flowchart illustrating a method for generating a unique identifier associated with a modification of a model according to some embodiments of the present disclosure; Fig. 12 a flowchart illustrating a method for using a unique identifier to provide information associated with a model, according to some embodiments of the present disclosure; Fig. 13 a flowchart illustrating a method for using a unique identifier to verify a model according to some embodiments of the present disclosure; Fig. 14-15 Flowcharts illustrating procedures for updating model maps with information describing modifications to models according to some embodiments of the present disclosure; Fig. 16 a block diagram of an exemplary computing device suitable for use in the implementation of at least some embodiments of the present disclosure; and Fig. 17 a block diagram of an exemplary data center suitable for use in the implementation of at least some embodiments of the present disclosure. DETAILED DESCRIPTION

[0020] Systems and procedures for tracking and verifying modifications to machine learning models are disclosed. For example, a system may provide models (e.g., machine learning models, neural networks, classifiers, etc.) using one or more techniques, such as managing one or more repositories, libraries, archives, databases, and / or similar resources. For instance, the system may provide a model repository that allows users to at least store, share, manage, access, modify, and / or use models. As described herein, models can be modified (e.g., updated, modified, optimized, extended, etc.) using one or more techniques.For example, a model can be modified by at least fine-tuning it to perform a specific task, by optimizing the model to run using specific hardware components (e.g., specific processor types, specific memory types, etc.), by tuning the model with hyperparameters, by quantizing the model, and / or by performing any other type of model modification technique.

[0021] In a specific example, the system(s) can receive an initial version of a model and then store it using a model repository. One or more initial users can then use one or more initial computing devices to access the initial version of the model in the model repository, modify the initial version of the model using one or more techniques to generate a second version of the model, and then upload the second version of the model back to the model repository.Next, one or more secondary users can use one or more secondary computing devices to access the first or second version of the model in the model repository, modify the first or second version of the model using one or more techniques to create a third version of the model, and then upload the third version of the model back to the model repository. These processes can then be repeated further, so that the model repository stores versions of the model that have been modified by different users using different techniques. For this reason, it may be important for the system(s) to provide mechanisms to at least track and / or verify modifications made to the models stored in the model repository.

[0022] As such, when a model is modified into a new version, the system(s) may generate, receive, obtain and / or retrieve data representing features associated with the model, modifications made to the model, information for users who made the modifications to the model, and / or other information associated with the new version of the model.In some examples, the data might represent, for instance, an architecture associated with the model, one or more datasets used to train the model, one or more risk scores associated with the model, one or more licenses associated with the model, one or more performance scores associated with the model, hardware for which the model was optimized, one or more changes made to the model, the identity of a user who made the modifications, and / or the date and / or time at which the modifications were made. In some examples, the system(s) may automatically generate, receive, obtain, and / or retrieve the data during and / or after the model modification.In some examples, the system(s) can generate, receive, obtain and / or retrieve the data based on one or more events, such as when the new version of the model is uploaded.

[0023] The system(s) can then process at least some of the data to generate structured information describing the features and / or modifications associated with the new version of the model. In some examples, the system can generate information (e.g., messages) that summarizes the changes made (e.g., fine-tuning to a new dataset, quantization for inference using a specific processor type, etc.), information detailing the modifications to the model (e.g., architectural changes, changes to risk assessment figures, license changes, etc.), and information describing performance improvements associated with the model (e.g., accuracy improvements, accuracy improvements on specific hardware, etc.).), information describing the date on which the modifications were made, information describing the user(s) who made the modifications, and / or information describing any other features and / or modifications associated with the model.

[0024] The system(s) can then generate a unique identifier that tracks at least the modifications associated with the new version of the model. As described herein, the unique identifier can include, but is not limited to, a checksum, a hash, a code, an alphabetic identifier, a numeric identifier, an alphanumeric identifier, and / or any other type of identifier that can be used to track the modifications associated with the model. For example, the system(s) can use one or more encoders to encode at least some of the structured information that describes the features and / or modifications associated with the new version of the model.In some examples, the encoder(s) can encode the structured information into a compressed format and / or a binary format, which can also be referred to as encoded information. The system(s) can then combine the encoded information with the model's configuration to generate the unique identifier associated with the model. For example, the system(s) can use one or more hashing algorithms (e.g., SHA-256, Blake3, etc.) to generate a checksum based on the combined encoded information and the model's configuration. By performing such processes, the unique identifier can represent at least the information describing the features and / or modifications associated with the new version of the model.

[0025] The system(s) can then store the unique identifiers and / or data associated with the models using one or more techniques. In some examples, the system can store the unique identifiers and / or data along with the models, such as using the model repository. Additionally or alternatively, in some examples, the system can send the unique identifiers, data, and / or models to one or more external platforms that are also used for storing and / or delivering the models. In other words, the system can synchronize the modifications made to the models with the other external platforms.

[0026] As described herein, the system(s) and / or one or more other computing devices can then use the unique identifiers to perform one or more tasks. For example, the system(s) and / or the computing device(s) can use a unique identifier associated with a model to, at a minimum, verify the integrity of the model. As described herein, in some examples, the system(s) and / or the computing device(s) can verify the integrity of the model by comparing a newly generated unique identifier with the unique identifier associated with the model. For example, the system(s) can provide the model, the data, and the unique identifier to a computing device, such as in response to receiving a request.The computing device(s) can then use one or more of the techniques described herein to generate a new unique identifier associated with the model. Furthermore, the computing device(s) can verify the integrity of the model if the unique identifiers match, and / or not verify the integrity of the model if the unique identifiers do not match. This is because, in some examples, the unique identifiers will no longer match if the model has been further modified (e.g., manipulated, such as by performing unauthorized modifications).

[0027] Furthermore, the system(s) can use unique identifiers to provide information associated with available models (e.g., models stored in the model repository). In one example, the system can use a unique identifier associated with a version of a model to retrieve information describing the features and / or modifications associated with that model. In some examples, such as when the unique identifier includes a checksum, the system can retrieve the information by decoding the checksum using one or more decoders. The system can then provide the information to one or more users.In this way, the user(s) can identify important information associated with the version of the model, such as the modifications made to the model, the modified features of the model, the impact of the modifications on the model's performance, who made the modifications, when the modifications were made, and / or similar information.

[0028] In a second example, the system(s) can use unique identifiers associated with multiple versions of a model to retrieve information describing features and / or modifications associated with one or more (e.g., each) of the versions. The system can then provide this information to one or more users, providing a clear audit trail of the model's history. For example, the user can use the information to identify changes that occurred at each modification step associated with the model. Furthermore, the user can use the information to identify a version of the model to retrieve for performing one or more tasks.For example, if the latest version of the model contains modifications that need to be reverted, the user(s) can use the information to identify in which version the modifications were introduced and / or retrieve the previous version of the model that does not contain the modifications.

[0029] In some examples, the system(s) may also use other techniques to track modifications made to the models. For example, the system(s) may generate and / or update model maps associated with the models. As described herein, model maps may initially contain fields that provide initial information associated with the models, such as fields for attributes, model architectures, intended use cases, out-of-scope applications, inputs, outputs, expected users, model performance for different groups, training, constraints, computational requirements, and / or similar information.Furthermore, an attribute may include, but is not limited to, a name and / or identifier of the model, one or more names and / or identifiers of one or more datasets used to train the model, one or more sizes of the dataset(s), a number of epochs used for training, a license type associated with the model, one or more risk scores associated with the model, one or more bias scores associated with the model, one or more losses associated with the model, and / or any other type of attribute.

[0030] The system(s) can then update the model cards to include information associated with modifications made to the models. For example, the system(s) can modify the model cards to include additional fields representing information related to the modifications, such as the types of modifications, the unique identifiers assigned to the models, the dates of the modifications, the performance information associated with the modifications (e.g., accuracy scores, precision scores, and recognition scores, etc.), the date the modified models are deployed, the hardware for which the models are optimized, the performance characteristics associated with the models running on the hardware (e.g., throughput, latency, power consumption, memory usage, etc.), and / or other information associated with the modifications.

[0031] For example, to update a model map for a model, the system(s) can first determine that the model is being updated and / or has been updated to generate a new version of the model. In some examples, the system can determine that the model has been updated based on the unique identifier associated with the new version of the model. For example, the system can determine that the unique identifier is new, so that the unique identifier indicates that the modification has been made. In some examples, the system can determine that the model has been updated by determining that a user is using a computing device to modify the model and / or has previously used the computing device to modify the model.In some examples, the system(s) can determine that the model has been updated based on the new version of the model uploaded to the system(s) (and / or another external repository). While these are just a few examples of how the system(s) can determine that the model has been updated, in other examples the system(s) can use additional and / or alternative techniques.

[0032] The system(s) can then determine additional information associated with the modifications made to the model. In some examples, the system can determine at least some of the additional information using the unique identifier. For example, the system can decode the unique identifier to obtain the structured information associated with the new version of the model. In some examples, the system can determine at least some of the additional information by using the data representing the features associated with the model, the modifications made to the model, and / or information about the users who made the modifications to the model, where the data was originally obtained to generate the unique identifier.In some examples, the system(s) can determine at least some of the additional information based on running one or more tests associated with the model, for example, to determine performance information associated with the new version of the model. While these are only some examples of how the system(s) can determine the additional information associated with modifications to the model, in other examples the system(s) can use additional and / or alternative techniques.

[0033] The system(s) can then use the additional information to update the model map associated with the model. For example, the system(s) can identify the model map associated with the model, such as by using a model identifier, a model map identifier, and / or another type of identifier. The system(s) can then incorporate at least some of the additional information into the model map. Furthermore, these processes can be repeated as the model is modified to generate further versions of the model. For example, if the model is optimized for ten different types of processors, the system(s) can modify the model map to include additional information associated with each of the model optimizations.Furthermore, these processes can be repeated to update the model maps for any number of models.

[0034] In some examples, the system(s) can generate a single model map, which is then updated with the additional information associated with each modification to a model. In other examples, however, the system can generate multiple model maps associated with the modifications made to the model. In one example, the system can generate an initial model map containing the initial information associated with a first version of the model and the additional information associated with a first modification to generate a second version of the model; a second model map containing the initial information associated with the first version of the model and the additional information associated with a second modification to generate a third version of the model; and so on.In a second example, the system(s) can generate a first model card containing first modifications for a first chain of modifications used to generate versions of a model, a second model card containing second modifications for a second chain of modifications used to generate additional versions of the model, and / or so on.

[0035] As described herein, the system(s) can then perform one or more processes using the model cards. In some examples, and similarly to the unique identifiers, the system can store the model cards locally in association with the models and / or send the model cards to one or more external platforms that are also used for storing and / or providing the models. In other words, the system can synchronize updates to the model cards with the external platforms. In some examples, the system can also provide the model cards to the users of the models.In this way, users are able to identify important information associated with the models, such as the modifications made to the models, the characteristics of the models that were modified (what hardware the models were optimized for to run on, etc.), the impact of the modifications on the performance of the models, who made the modifications, when the modifications were made, and / or similar information.

[0036] In some examples, the system(s) may provide one or more mechanisms for viewing and / or browsing the model cards. Since a model card can be updated to include information about numerous modifications (e.g., hundreds of modifications, etc.) associated with a model, the system may create a user interface that allows users to quickly browse the model card to identify specific information. In a first example, where a model card contains a text file, the user interface may include one or more interface elements (e.g., buttons, search bars, sliders, etc.) that a user can use to select different versions of the model. The user interface may then provide information related to the selected versions of the model.In a second example, the system(s) can convert a text file associated with the model map into a format that can be searched by users.

[0037] The systems and procedures described herein may be used without restriction by non-autonomous vehicles or machines, semi-autonomous vehicles or machines (e.g., in one or more adaptive driver assistance systems (ADAS)), autonomous vehicles or machines, steered and unsteered robots or robotic platforms, warehouse vehicles, all-terrain vehicles, vehicles coupled to one or more trailers, flying vehicles, boats, shuttles, emergency response vehicles, motorcycles, electric or motorized bicycles, aircraft, construction vehicles, underwater vehicles, drones and / or other types of vehicles.Furthermore, the systems and methods described herein can be used for a variety of purposes, including but not limited to machine control, machine movement, machine driving, synthetic data generation, model training, perception, augmented reality, virtual reality, mixed reality, robotics, security and surveillance, simulation and digital twinning, autonomous or semi-autonomous machine applications, deep learning, environmental simulation, object or actor simulation and / or digital twinning, data center processing, conversational AI, light transport simulation (e.g., ray tracing, path tracing, etc.), collaborative content creation for 3D assets, cloud computing, and / or other suitable applications.

[0038] The disclosed embodiments can be included in a variety of different systems, such as automotive systems (e.g., a control system for an autonomous or semi-autonomous machine, a perception system for an autonomous or semi-autonomous machine), systems implemented using a robot, air systems, media systems, boat systems, intelligent area monitoring systems, systems for performing deep learning operations, systems for performing simulation operations, systems for performing digital twin operations, systems implemented using an edge device, systems implementing language models (e.g.,small language models (SLMs (including mini and tiny language models), large language models (LLMs)), systems implementing one or more vision language models (VLMs), systems implementing one or more multimodal language models, systems using or employing one or more inference microservices, systems that include one or more machine learning models in a service or microservice along with a virtualization package (e.g.,a container) at the OS level, systems that include or use one or more virtual machines (VMs), systems for performing synthetic data generation operations, systems that are at least partially implemented in a data center, systems for performing conversational AI operations, systems for performing light transport simulations, systems for performing collaborative content creation for 3D assets, systems for performing generative AI operations, systems that are at least partially implemented using cloud computing resources, and / or other types of systems.

[0039] With reference to Fig. 1, illustrated Fig. 1 An example of a process 100 for the use of unique identifiers for tracking and verifying modifications to models according to some embodiments of the present disclosure. It is pointed out that this and other arrangements described herein are presented only as examples. Other arrangements and elements (e.g., machines, interfaces, functions, sequences, groupings of functions, etc.) may be used in addition to or instead of those shown, and some elements may be omitted entirely. Furthermore, many of the elements described herein are functional entities that may be implemented as single or distributed components, or in conjunction with other components, in any suitable combination and at any suitable location. Various functions described herein, performed by entities, may be executed by hardware, firmware, and / or software.For example, different functions can be performed by a processor that executes instructions stored in the main memory.

[0040] The process 100 can contain one or more systems 102 that store models 104 in one or more databases 106. As described herein, a model 104 can contain, but is not limited to, a machine learning model, a neural network, a classifier, and / or the like. Furthermore, the system(s) 102 can contain, include, and / or manage one or more repositories, libraries, archives, databases, and / or the like associated with the models 104. For example, the system(s) 102 can provide a model repository that allows users to at least store, share, manage, access, modify, and / or use the models 104.As described herein, Model 104 can be modified—such as updated, modified, optimized, and / or extended—using one or more techniques. For example, a Model 104 can be modified by at least fine-tuning it to perform a specific task, by optimizing it to run using specific hardware components (e.g., processors, memory, etc.), by tuning it with hyperparameters, by quantizing it, and / or by employing any other type of model modification technique.

[0041] In a specific example, the system(s) 102 can receive an initial version of a model 104 and then store it using the model repository. One or more first users can then use one or more first computing devices 108 to access the initial version of the model in the model repository, modify the initial version of the model 104 to generate a second version of the model 104, and then upload the second version of the model 104 back to the model repository. In some examples, the first user(s) may be authorized and / or certified users who are able to modify the model 104 and / or upload the second version of the model 104 back to the model repository.In some examples, the first user(s) can modify the first version of Model 104 using any technique, such as fine-tuning Model 104 to perform a new task, and / or optimizing Model 104 for execution using specific hardware.

[0042] Next, one or more second users can use one or more second computing devices 108 to access the first version of Model 104 and / or the second version of Model 104 in the model repository, modify the first version of Model 104 and / or the second version of Model 104 to create a third version of Model 104, and then upload the third version of Model 104 back to the model repository. In some examples, the second user(s) may be authorized and / or certified users who are able to modify Model 104 and / or upload the third version of Model 104 back to the model repository.In some examples, the second user(s) can modify the first version of Model 104 and / or the second version of Model 104 using any technique, such as fine-tuning Model 104 to perform a new task and / or optimizing Model 104 for execution using specific hardware. These processes can then be repeated, so that the model repository stores versions of Model 104 that have been modified by different users using different techniques.

[0043] For example, it illustrates Fig. 2 An example of storing different versions of a model that have been modified, according to some embodiments of the present disclosure. As shown, the model repository can store a first version of a model 202(1) (e.g., an initial version of the model), a second version of the model 202(2) generated by modification 204(1) of the first version of the model 202(1), a third version of the model 202(3) generated by modification 204(2) of the second version of the model 202(2), and a fourth version of the model 202(4) generated by modification 204(3) of the third version of the model 202(3). Additionally, the model repository can store a fifth version of model 202(5), which is also generated by modification 204(4) of the second version of model 202(2), and a sixth version of model 202(6), which is generated by modification 204(5) of the fifth version of model 202(5).Users can use computing devices to communicate with the model repository to access, download, retrieve, receive, modify and / or use one or more versions of Model 202(1)-(6).

[0044] Referring again to the example of Fig. 1. Since the models 104 can be modified using different techniques and / or by different users, the system(s) 102 can provide one or more mechanisms to track, identify, and / or verify the modifications associated with the models 104. For example, the process 100 can include the use of one or more modification components 110 configured to generate, receive, obtain, and / or retrieve model data 112 representing features associated with the models 104, modifications that have taken place with respect to the models 104, information for users who made the modifications to the models 104, and / or other information associated with the models 104.In some examples and for one version of Model 104, the Model Data 112 may, for example, represent an architecture associated with Model 104, one or more datasets used to train Model 104, one or more risk scores associated with Model 104, one or more licenses associated with Model 104, one or more performance scores associated with Model 104, hardware for which Model 104 was optimized, one or more changes made to Model 104, the identity of a user who made the modifications, and / or the date and / or time at which the modifications were made. In some examples, the modification component(s) 110 may automatically generate, receive, obtain, and / or retrieve the Model Data 112 during and / or after the modification of Model 104.In some examples, the modification component(s) 110 can generate, receive, obtain and / or retrieve the model data 112 based on one or more events, such as when the new version of the model 104 is uploaded.

[0045] For example, it illustrates Fig. 3 An example of obtaining data representing features and / or modifications associated with a model, according to some embodiments of the present disclosure. In the example of Fig. 3. The modification component(s) 110 may receive at least model data 302 (which may contain and / or be similar to model data 112) that are associated with the sixth version of model 202(6).As shown, the model data 302 can represent at least one architecture 304 of the sixth version of the model 202(6), one or more datasets 306 used to train the sixth version of the model 202(6), one or more risk score numbers 308 associated with the sixth version of the model 202(6), one or more licenses 310 associated with the sixth version of the model 202(6), one or more performance score numbers 312 associated with the sixth version of the model 202(6), hardware 314 for which the sixth version of the model 202(6) was optimized, and / or other information 316 associated with the features and / or modifications associated with the sixth version of the model 202(6). In some examples, the modification component(s) 110 can receive similar model data that is associated with one or more of the other versions of the model 202(1)-(5).

[0046] Referring again to the example of Fig. 1. The process 100 can contain one or more information components 114 that use at least some part of the model data 112 to generate structured information 116 associated with the models 104. As described herein, structured information 116 associated with a version of a model 104 can generate information (e.g., messages) that summarizes the changes made (e.g., fine-tuning to a new dataset, quantization for inference using a specific processor type, etc.), information that details the modifications to the model 104 (e.g., changes to the architecture, changes to risk assessment figures, changes to licenses, etc.), and information that describes performance improvements associated with the model 104 (e.g., accuracy improvements, accuracy improvements on specific hardware, etc.).), information describing the date on which the modifications were made, information describing the user(s) who made the modifications, and / or information describing any other features and / or modifications associated with Model 104, but not limited to.

[0047] For example, it illustrates Fig. 4 An example of the generation of structured information associated with a version of a model, according to some embodiments of the present disclosure. As shown, the information component(s) 114 can process at least the model data 302 associated with the sixth version of the model 202(6) and / or model data 402 associated with one or more of the other versions of the model 202(1)-(5). At least on the basis of this processing, the information component(s) 114 can generate structured information 404 (which may contain and / or be similar to the structured information 116) associated with the sixth version of the model 202(6). In the example of Fig. 4. The structured information 404 can contain at least one commit message 406, indicating that a modification involved fine-tuning the model on a new dataset; modification details 408, containing information about the new dataset; performance improvements 410, indicating that the model's accuracy increased by 2%; and modification details 412, specifying a user identifier and a date and time the modification occurred. However, in other examples, the structured information 404 can also contain additional and / or alternative types of information.

[0048] Referring again to the example of Fig. 1. The process 100 can contain one or more encoding components 118 that encode at least some of the structured information 116 to produce encoded information 120. As described herein, the encoding component(s) 118 can contain and / or use any type of encoder to produce the encoded information 120. The encoded information 120 can, for example, contain Unicode, ISO-8859-1, UTF-16, code page, label encoding, Base64 encoding, and / or any other type of encoded text. While the example of Fig. 1. As illustrated by the process 100 containing the coding component(s) 118 that generate the coded information 120, the process 100 may not contain the coding component(s) 118 that generate the coded information 120 in other examples.

[0049] The process 100 can contain one or more identifier components 122 that process at least some part of the coded information 120 (and / or, in some examples, the structured information 116) to generate unique identifiers 124 associated with the models 104, the unique identifiers 124 being stored in one or more databases 126. As described herein, a unique identifier 124 can contain, but is not limited to, a checksum, a hash, a code, an alphabetic identifier, a numeric identifier, an alphanumeric identifier, and / or any other type of identifier that can be used to track modifications associated with a model 104. Furthermore, in some examples, the identifier component(s) 122 can contain one or more hash algorithms (e.g., SHA-256, Blake3, etc.).) contain and / or use to generate checksums based at least on the coded information 120. In some examples, the identifier component(s) 122 can combine the coded information 120 with additional information when generating the unique identifiers 124. For example, the identifier component(s) 122 can combine the coded information 120 with information describing configurations 128 of the models 104, and then use the combined information to generate the unique identifiers 124.

[0050] In some examples, the identifier component(s) 122 can generate unique identifiers 124 that are associated with each of the versions of the models 104. For example, each time a new version of a model 104 is uploaded to the database(s) 106, the identifier component(s) 122 can generate a unique identifier 124 that is associated with the new version of the model 104. However, in some examples, the identifier component(s) 122 can generate one or more unique identifiers 124 that are associated with one or more specific versions of the models 104. For example, the identifier component(s) 122 can generate unique identifiers 124 for specific types of modifications, such as when models 104 are optimized for specific hardware.

[0051] For example, it illustrates Fig. 5 An example of the generation of unique identifiers that are assigned to the different versions of Model 202(1)-(6) according to some embodiments of the present disclosure. As shown, the identifier component(s) 122 can generate at least one first unique identifier 502(1) associated with the first version of the model 202(1), one second unique identifier 502(2) associated with the second version of the model 202(2), one third unique identifier 502(3) associated with the third version of the model 202(3), one fourth unique identifier 502(4) associated with the fourth version of the model 202(4), one fifth unique identifier 502(5) associated with the fifth version of the model 202(5), and one sixth unique identifier 502(6) associated with the sixth version of the model 202(6).As described herein, the unique identifiers 502(1)-(6) can be associated with features and / or modifications that are linked to versions of model 202(1)-(6). For example, the sixth unique identifier 502(6) can represent information linked to the fifth modification 204(5) of the fifth version of model 202(5) to generate the sixth version of model 202(6). The identifier component(s) 122 can, for example, generate the sixth unique identifier 502(6) using the structured information 404.

[0052] Referring again to the example of Fig. 1. Process 100 can include the local storage of the models 104 and / or the unique identifiers 124 on the system(s) 102. For example, the system(s) can store the unique identifiers 124 in association with the models 104, so that the system(s) 102 is / are later able to identify the unique identifiers 124 when one or more events occur. Furthermore, in some examples, the process 100 may include the provision of at least some of the models 104 to external repositories 130, wherein the models 104 provided to the external repositories 130 are represented by models 132, and / or the provision of at least some of the unique identifiers 124 to the external repositories 130, wherein the unique identifiers 124 provided to the external repositories 130 are represented by unique identifiers 134.In other words, the system(s) 102 can synchronize the modifications to the models 132 with the external repositories 130 and simultaneously provide the external repositories 130 with the assigned unique identifiers 134. In this way, the external repositories 130 can perform one or more of the processes described herein with respect to the system(s) 102.

[0053] As described herein, the system(s) 102, the computing devices 108, and / or the external repositories 130 can use the unique identifiers 124 associated with the models 104 to perform one or more tasks. For example, the process 100 can use one or more decoding components 136 to decode the unique identifiers 124. As described herein, the decoding component(s) 136 can decode unique identifiers 124 based on the occurrence of one or more events. For example, the decoding component(s) 136 can decode unique identifiers 124 based on receiving requests from computing devices 108, receiving the unique identifiers 124 from computing devices 108 (e.g.,After the unique identifiers 124 have been provided to the computing devices 108 with associated models 104, in response to the provision of the associated models 104 to computing devices 108 and / or based on any other type of event, the decoding component(s) 136 may contain and / or use any type of decoder to perform one or more of the processes described herein.

[0054] Based on receiving a request to decode a unique identifier 124 associated with a model 104, the decoding component(s) 136 can, for example, decode the unique identifier 124 to generate and / or retrieve the structured information 116 associated with the model 104. In some examples, the decoding component(s) 136 can also decode one or more additional unique identifiers 124 associated with one or more additional versions of the model 104. In such examples, the decoding component(s) 136 can thus generate and / or retrieve the structured information 116 representing a unique audit trail associated with the model 104.The system(s) 102 can then provide the structured information 116 associated with the model 104 and / or the structured information 116 associated with one or more additional versions of the model 104 to a computing device 108. Based on the receipt of the structured information 116, the computing device 108 can then provide the structured information 116 to a user (e.g., display or output audio in association with it, etc.).

[0055] For example, it illustrates Fig. 6 An example of providing structured information associated with different versions of a model, according to some embodiments of the present disclosure. As shown, a user interface 602 can include at least first structured information 604(1) associated with the first modification 204(1) of the first version of the model 202(1) to generate the second version of the model 202(2), second structured information 604(2) associated with the second modification 204(2) to the second version of the model 202(2) to generate the third version of the model 202(3), and third structured information 604(3) associated with the third modification 204(3) to the third version of the model 202(3) to generate the fourth version of the model 202(4).For example, the first structured information 604(1) indicates that the first modification 204(1) included a fine-tuning of the model for carrying out task 1, the second structured information 604(2) indicates that the second modification 204(2) included a fine-tuning of the model for carrying out task 2, and the third structured information 604(3) indicates that the third modification 204(3) included an optimization of the model for specific hardware (e.g., a specific type of processor, a specific type of memory, etc.).

[0056] Referring again to the example of Fig. 1. The user can then use the structured information 116 to perform one or more tasks. For example, when the user determines whether to use a version of Model 104 and / or uses the version of Model 104, the user can use the structured information 116 to identify various modifications made to Model 104, the performance of the version of Model 104, the performance of the version of Model 104 on specific hardware, the performance of the version of Model 104 in performing a specific task, the users who made the modifications, the times when the modifications to Model 104 were made, and / or any other information that is important to the user.Using this information, the user can then decide whether to use the version of Model 104 - for performing one or more tasks and / or for further modifications.

[0057] A second example: The user can use the structured information 116 to identify a history of modifications made to model 104, detailing how and what changes were made to create each version of model 104. Based on this structured information 116, the user can then select a version of model 104 to use. For example, if the user determines that the modifications to model 104 were accurate, appropriate, and / or necessary, they can choose to use the latest version of model 104 to perform one or more tasks. However, if the user determines that one or more of the modifications were not accurate, appropriate, and / or necessary, they can choose to revert the modification(s).As such, the user can further use the structured information 116 to identify a version of model 104 that does not contain the modification(s) – such as the version of model 104 on which the modification(s) were made – and use this version of the model to perform one or more tasks. In other words, the user can revert to a version of model 104 that existed before the modification(s).

[0058] For example, and in the example of Fig. 6. The user can use the user interface 602 to determine that the third modification 204(3) included the optimization of the model for the specific hardware. However, the user may want to run the model using a different type of hardware. Thus, the user can use the structured information 604(1)-(3) to determine that they want to use the third version of the model 202(3), which was not optimized for the specific hardware, instead of the fourth version of the model 202(4), which was optimized for the specific hardware.

[0059] In some examples, the unique identifiers 124 can also be used to verify the integrity of the models 104, for example, by checking that the models 104 have not been tampered with and / or that no unauthorized modifications have been made to the models 104. For example, the system(s) can provide a model 104, along with an associated unique identifier 124, to a computing device 108. The computing device 108 can then use the unique identifier 124 to verify the integrity of the model 104. For example, the computing device 108 can compare the received unique identifier 124 with the unique identifier 124 stored on the system(s) 102.Based at least on the comparison, the computing device 108 can verify the integrity of the model 104 if the unique identifiers 124 match, and / or fail to verify the integrity of the model 104 if the unique identifiers 124 do not match. This is because, in some examples, the unique identifiers 124 no longer match if the model 104 has been further modified (e.g., manipulated, such as by performing unauthorized modifications).

[0060] A second example: The computing device 108 can perform similar processes to the system(s) 102 to generate a new unique identifier 124 that is associated with the model 104. The computing device 108 can then compare the received unique identifier 124 with the new unique identifier 124. Based at least on this comparison, the computing device 108 can re-verify the integrity of the model 104 if the unique identifiers 124 match, and / or not verify the integrity of the model 104 if the unique identifiers 124 do not match. This is because, in some examples, the new unique identifiers 124 do not match the received unique identifier 124 if the model 104 has been further modified (e.g., manipulated, such as by performing unauthorized modifications).While these are only two exemplary techniques of how a computing device 108 can use a unique identifier 124 to verify the integrity of a model 104, in other examples a computing device 108 may use additional and / or alternative techniques to verify the integrity of the model 104 using the unique identifier 124.

[0061] While the example of Fig. 1. As illustrated by the system(s) 102 containing the modification component(s) 110, the information component(s) 114, the encoding component(s) 118, the identifier component(s) 122, and the decoding component(s) 136, in other examples the computing device(s) 108 and / or the external repositories 130 may contain the modification component(s) 110, the information component(s) 114, the encoding component(s) 118, the identifier component(s) 122, and the decoding component(s) 136. The process 100, for example, can be carried out using the system(s) 102, the computing device(s) 108, and / or the external repositories 130. Furthermore, a component (e.g.,the modification component(s) 110, the information component(s) 114, the encoding component(s) 118, the identifier component(s) 122 and the decoding component(s) 136) software, hardware, a machine learning model, a neural network, a module, a classifier, an algorithm, an encoder, a decoder, a program, an application and / or any other type of processing component.

[0062] Fig. Figure 7 illustrates an example of a Process 700 for automatically updating model maps to include additional information for modifications to models, according to some embodiments of the present disclosure. It should be noted that this and other arrangements described herein are presented only as examples. Other arrangements and elements (e.g., machines, interfaces, functions, sequences, groupings of functions, etc.) may be used in addition to or instead of those shown, and some elements may be omitted entirely. Furthermore, many of the elements described herein are functional entities that can be implemented as single or distributed components, or in conjunction with other components, in any suitable combination and location.Various functions described herein, which are performed by entities, can be executed by hardware, firmware, and / or software. For example, various functions can be performed by a processor executing instructions stored in memory.

[0063] For example, process 700 can contain one or more detection components 702 that determine that models 104 are being modified, that models 104 have been modified, that new versions of models 104 have been created, and / or that new versions of models 104 have been stored in database(s) 106. In some examples, the detection component(s) 702 can use a unique identifier 124 associated with the new version of model 104 to determine that a model 104 has been modified. A first example: The detection component(s) 702 can determine that the unique identifier 124 is being created and / or recreated, as by using process 100 from the example of Fig. 1. For example, as described herein, unique identifiers 124 can be generated for models 104 when the models 104 are modified in order to track and / or verify the modifications made to the models 104.

[0064] A second example: The recognition component(s) 702 can determine that the unique identifier 124 is new in the database(s) 126 and / or differs from one or more other unique identifiers 124 associated with one or more previous versions of the model 104. Since the unique identifiers 124 are generated, for example, as described herein, using information that describes features associated with the models 104 and / or modifications made to the models 104, each unique identifier 124 is different from the other unique identifiers 124. When the recognition component(s) 702 detects a new unique identifier 124 in the database(s) 126, the recognition component(s) 702 can determine that the model 104 associated with the unique identifier 124 has been modified.While these are only two exemplary techniques of how the detection component(s) 702 can use the unique identifiers 124 to determine that the models 104 have been modified, in other examples the detection component(s) 702 can use additional and / or alternative techniques.

[0065] In some examples, the detection component(s) 702 may use one or more additional and / or alternative techniques to determine that a Model 104 is being modified and / or has been modified. For example, the detection component(s) 104 may determine that a Model 104 is being modified and / or has been modified by determining that a computing device 108 is accessing the Model 104, determining that a computing device 108 is modifying the Model 104, determining that a computing device 108 is uploading a new version of the Model 104 to the system(s) 102, receiving an indication that the Model 104 is being modified, receiving an indication that the Model 104 has been modified, receiving an indication that a new version of the Model 104 exists, and / or by using another technique.

[0066] The process 700 can then contain one or more decoding components 704 that process unique identifiers 124 to obtain modification information 706 associated with the modifications, where the modification information 706 can also be referred to as additional information. For example, if it is determined that a model 104 has been modified to create a new version of model 104, the decoding component(s) 704 can decode the unique identifier 124 associated with model 104 to retrieve the structured information 116 describing the features and / or modifications associated with model 104, where the modification information 706 can contain at least some of the structured information 116.As described herein, the decoding component(s) 704 may contain and / or use any type of decoder to perform one or more of the processes described herein. As such, process 700 may include the use of the unique identifier 124 associated with model 104 to determine that model 104 has been modified, and then further use the unique identifier 124 to obtain the modification information 706 describing the modifications associated with model 104.

[0067] For example, it illustrates Fig. 8 An example of decoding a unique identifier to obtain additional information associated with a modification of a model, according to some embodiments of the present disclosure. In the example of Fig. 8. The detection component(s) 702 may first use the sixth unique identifier 502(6) associated with the sixth version of model 202(6) to determine that the model has been modified. The decoding component(s) 704 may decode the sixth unique identifier 502(6) to obtain modification information 802 describing the fifth modification 204(5) made with respect to the sixth version of model 202(6). For example, the modification information 802 may include at least the commit message 406, the modification details 408, the performance improvements 410, and / or the modification details 412 from the structured information 404 associated with the sixth version of model 202(6).In other examples, however, the modification information 802 may contain additional and / or alternative information associated with the fifth modification 204(5) of the sixth version of the model 202(6).

[0068] Referring again to the example of Fig. The process 700 may include one or more test components 708 that process the models 104 to generate performance information 710 associated with the models 104, where the performance information 710 may also be referred to as additional information. For example, if it is determined that a model 104 has been modified, the test component(s) 708 may perform one or more tests on the new version of the model 104 to determine the performance information 710. As described herein, the performance information 710 associated with the model 104 may include, but is not limited to, an accuracy rating score, a precision rating score, a retrieval rating score, a loss rating score, throughput, latency, memory usage, and / or any other type of performance metric associated with the model 104.Furthermore, the test component(s) 708 can perform any type of test with the model 104 to determine the performance information 710.

[0069] The test component(s) 708 can, for example, input instances of input data into the model 104 for processing. The input data instances can include, for example, images if the model 104 contains a vision model, audio data representing speech if the model 104 contains a speech model 104, and / or any other type of input data corresponding to a type of model 104. The test component(s) 708 can then receive outputs based at least on the model 104 processing the input data and use the outputs to determine the performance information 710 associated with the model 104. For example, the test component(s) 708 can determine at least one accuracy rating score associated with the model 104 by comparing the outputs of the model 104 with the basic truth outputs associated with the input data.While this is only one exemplary technique of how the test component(s) 708 can test the model 104 to determine the performance information 710, in other examples the test component(s) 708 can use additional and / or alternative techniques.

[0070] For example, it illustrates Fig. Figure 9 is an example of testing a model for determining performance data according to some embodiments of the present disclosure. As shown, the test component(s) 708 can input data 902 into the sixth version of the model 202(6). As described herein, the input data 902 can be of a data type for which the sixth version of the model 202(6) is trained. The input data 902 can, for example, include image data, video data, audio data, text data, sensor data, and / or any other type of data. The sixth version of the model 202(6) can then process the input data 902 and, at least on the basis of the processing, generate output data 904.The test component(s) 708 can then use the output data 904 together with basic truth data 906 corresponding to the input data 902 to determine performance information 908 (which may contain and / or be similar to the performance information 710) associated with the sixth version of model 202(6).

[0071] The verification component(s) 708 can, for example, determine an accuracy rating assigned to the sixth version of model 202(6), at least by comparing the output data 904 with the basic truth data 906. The verification component(s) 708 can, for example, determine the accuracy rating based on the number of correct outputs determined by comparing the output data 904 with the basic truth data 906 and the total number of outputs. Additionally, in some examples, the test component(s) 708 may determine additional performance information 908 associated with the sixth version of Model 202(6), based at least on the tests, such as a precision score, a retrieval score, a loss score, throughput, latency, memory utilization, and / or any other type of performance metric associated with the sixth version of Model 202(6).

[0072] Referring again to the example of Fig. 7. The process 700 can contain one or more update components 712 that receive at least the modification information 706 and the performance information 710. In some examples, the update component(s) 712 can receive additional supplementary information associated with modifications to the models 104, such as information provided by the model data 112 from the example of Fig. The process 100 can then include the update component(s) 712, which use at least some of the additional information to update model cards 714 associated with the models 104, the model cards 714 being stored in one or more databases 716. As described herein, the update component(s) 712 can use any technique to update the model cards 714 using the additional information.

[0073] For example, model maps 714 may initially contain fields that provide initial information associated with the first versions of the models 104, such as fields for attributes, model architectures, intended use cases, applications outside the scope, inputs, outputs, expected users, model performance for different groups, training, constraints, computational requirements and / or similar.Furthermore, an attribute can include, but is not limited to, a name and / or identifier of a Model 104, one or more names and / or identifiers of one or more datasets used to train the Model 104, one or more sizes of the dataset(s), a number of epochs used for training, a license type associated with the Model 104, one or more risk scores associated with the Model 104, one or more bias scores associated with the Model 104, one or more losses associated with the Model 104, and / or any other type of attribute. In some examples, the System(s) 102 can automatically generate the Model Cards 714 when they receive the first versions of the Model(s) 104.

[0074] The update component(s) 712 can then update the model cards 714 to include additional information associated with the modifications made to the models 104. For example, the update component(s) 712 can modify the model cards 714 to include additional fields representing information related to the modifications, such as the types of modifications, the unique identifiers 124 assigned to the models 104, the dates of the modifications, the performance information associated with the modifications (e.g., accuracy rating numbers, precision rating numbers, and recognition rating numbers, etc.), the date the modified models 104 are deployed, the hardware for which the models 104 are optimized, and the performance characteristics assigned to the models 104 that run on the hardware (e.g.,Throughput, latency, power consumption, memory usage, etc.), and / or other information associated with the modifications. In some examples, the update component(s) 712 can update the model cards 714 with each modification made to the models 104.

[0075] The update component(s) 712 can, for example, receive additional information associated with a new version of a model, such as the modification information 706, the performance information 710, and / or the information represented by the model data 112. The update component(s) 712 can also receive an identifier associated with the model 104 and / or an identifier associated with a model card 714 corresponding to the model 104. The update component(s) 712 can then use the identifier(s) to retrieve the model card 714 associated with the model 104. In some examples, the model card 714 may contain the output information associated with the model 104. In some examples, the model card 714 may also contain additional information associated with one or more previous modifications of the model 104.In each of the examples, the update component(s) 712 can then use at least some of the additional information to update the model card 714 so that it contains information describing the new version of the modified model 104.

[0076] For example, illustrate Fig. Figures 10A-10C provide an example of updating a model map to include additional information associated with modifications to a model, according to some embodiments of the present disclosure. As illustrated by the example of Fig. As shown in Figure 10A, a model card 1002 (which may contain and / or be similar to a model card 714) may initially contain information 1004 corresponding to the first version of the model 202(1). For example, the information 1004 may describe at least one version 1006 of the model, a unique identifier 1008 of the model (e.g., a name, etc.), an architecture 1010 associated with the model, one or more data sets 1012 used to initially train the model, and a performance 1014 of the model. However, in other examples, the information 1004 may contain additional and / or alternative fields describing additional and / or alternative features associated with the model.

[0077] Next, and as illustrated by the example of Fig. As shown in Figure 10B, the update component(s) 712 can then receive additional information 1016, which is associated with the first modification 204(1) of the first version of model 202(1), in order to generate the second version of model 202(2). The update component(s) 712 can then use at least some of the additional information 1016 to update the model card 1002 to describe the first modification 204(1) and / or the second version of model 202(2). For example, the additional information 1016, as shown, can include at least one version 1018, modification characteristics 1020 (e.g., a type of modification, a date of modification, an updated accuracy associated with the modification, a deployment date, etc.), deployment characteristics 1022 (e.g., a type of hardware, a throughput, a latency, a memory usage, etc.), a performance 1024 (e.g.,an accuracy rating score, a precision rating score, a retrieval score, etc.) and a unique identifier 1026 (e.g., the second unique identifier 502(2)) that is assigned to the second version of the model 202(2). In other examples, however, the additional information 1016 may contain additional and / or alternative fields that describe additional and / or alternative features assigned to the model.

[0078] Next, and as illustrated by the example of Fig. As shown in Figure 10C, the update component(s) 712 can then receive additional information 1028, which is associated with the second modification 204(2) of the second version of model 202(2), in order to generate the third version of model 202(3). The update component(s) 712 can then use at least some of the additional information 1028 to update the model card 1002 to describe the second modification 204(2) and / or the third version of model 202(3). For example, the additional information 1028, as shown, can include at least one version 1030, modification characteristics 1032 (e.g., a type of modification, a date of modification, an updated accuracy associated with the modification, a deployment date, etc.), deployment characteristics 1034 (e.g., a type of hardware, a throughput, a latency, a memory usage, etc.), a performance 1036 (e.g.,an accuracy rating score, a precision rating score, a retrieval score, etc.) and a unique identifier 1038 (e.g., the third unique identifier 502(3)) that is assigned to the third version of the model 202(3). In other examples, however, the additional information 1028 may contain additional and / or alternative fields that describe additional and / or alternative features assigned to the model.

[0079] This process can then be repeated to update model card 1002 with additional information associated with the third modification 204(3), the fourth version of model 202(4), the fourth modification 204(4), the fifth version of model 202(5), the fifth modification 204(5), and / or the sixth version of model 202(6). While the example of Fig. In examples 10A-10C, which describes updating model card 1002 to include the additional information associated with all modifications 204(1)-(5) and / or all modified versions of model 202(2)-(6), update component(s) 108 can, in other examples, generate and / or update multiple model cards associated with the model. For example, update component(s) 108 can generate a first model card containing information associated with modifications 204(1)-(3) and versions of model 202(1)-(4), and a second model card containing information associated with modifications 204(4)-(5) and versions of model 202(1) and 202(5)-(6).

[0080] Referring again to the example of Fig. In example 7, process 700 can include the fact that the update component(s) 712 outputs updated model cards 718 that are associated with the models 104. In some examples, system(s) 102 can then store the updated model cards 718 in association with the models 104, as in database(s) 716. In some examples, system(s) 102 can further send at least some of the updated model cards 718 to external repositories 130 for storage, where the updated model cards 718 sent to external repositories 130 can be represented by model cards 720. In other words, system(s) 102 can synchronize the updates of the updated model cards 718 with the external repositories 130.

[0081] In some examples, the system(s) 102 can then perform one or more operations using the updated model maps 718. For example, the system(s) 102 can generate a user interface 722 for a model 104, which contains at least part of the updated model map 718 associated with the model 104. As described herein, the system(s) 102 can generate the user interface 722 in response to one or more events, such as receiving a request for the model 104, receiving a request for a version of the model 104, receiving a request for information about the model 104, receiving a request for information about a version of the model 104, and / or any other event. The system(s) 102 can then send data representing the user interface 722 to at least one computing device 108.In this way, the computing device 108 can provide the user interface 722 to one or more users who can use the user interface 722 to identify information associated not only with the original version of the Model 104, but also with all modified versions of the Model 104.

[0082] The user(s) can, for example, use the 722 user interface to identify specific modifications made to the Model 104. A first example: If the user is looking for one or more versions of the Model 104 that are modified to perform a specific task, such as object recognition with respect to a particular object type, then the user(s) can use the 722 user interface to identify a version of the Model 104 that has been trained to perform that specific task. A second example: If the user is looking for one or more versions of the Model 104 that are optimized to run on specific hardware, such as a particular processor type, then the user(s) can use the 722 user interface to identify a version of the Model 104 that has been optimized for that specific hardware.A third example: If the user(s) are looking for one or more versions of a Model 104 that meet a specific performance requirement, such as an accuracy of 95% or more, then the user(s) can use the user interface 722 to identify a version of the Model 104 that meets the specific performance requirement.

[0083] As described herein, System(s) 102 can provide the updated model maps 718 using various formats in some examples. In a first example, System(s) 102 can generate an updated model map 718 using a text format, so that the user interface 722 presents a text file corresponding to the updated model map 718. In such an example, the updated model map 718 can still be searchable, such as by allowing users to enter text queries, by providing interface elements (e.g., tabs, buttons, etc.) that can be selected to navigate to different parts of the updated model map 718, and / or by using some other technique.A second example: System(s) 102 can generate an updated model map 718 using an interactive format, allowing the user interface 722 to interact with the updated model map 718. For example, users can select different parts of the updated model map 718, such as parts that correspond to additional information for various modifications to the model 104. Based on the selection of a part of the updated model map 718, users can then be provided with additional information corresponding to the selected part.

[0084] While the example of Fig. Figure 7 illustrates that the system(s) 102 contains the recognition component(s) 702, the decoding component(s) 704, the verification component(s) 708, and the update component(s) 712. In other examples, the computing device(s) 108 and / or the external repositories 130 may contain the recognition component(s) 702, the decoding component(s) 704, the verification component(s) 708, and the update component(s) 712. For example, the process 700 can be performed using the system(s) 102, the computing device(s) 108, and / or the external repositories 130. Furthermore, a component (e.g.,the recognition component(s) 702, the decoding component(s) 704, the verification component(s) 708 and the update component(s) 712) software, hardware, a machine learning model, a neural network, a module, a classifier, an algorithm, an encoder, a decoder, a program, an application and / or any other type of processing component.

[0085] With reference to Fig. Sections 11-15 comprise each block of procedures 1100, 1200, 1300, 1400, and 1500 described herein, a computational process that can be performed using any combination of hardware, firmware, and / or software. For example, various functions can be performed by a processor executing instructions stored in memory. Procedures 1100, 1200, 1300, 1400, and 1500 can also be embodied as computer-usable instructions stored on computer storage media. Procedures 1100, 1200, 1300, 1400, and 1500 can be provided by a standalone application, a service or hosted service (standalone or in combination with another hosted service), or a plug-in for another product, to name just a few. Furthermore, procedures 1100, 1200, 1300, 1400 and 1500 are exemplified with regard to the system of Fig. 1 and Fig. 7 described. However, these procedures 1100, 1200, 1300, 1400 and 1500 can additionally or alternatively be performed by any system or any combination of systems, including, but not limited to, the systems described herein.

[0086] Fig. Figure 11 illustrates a flowchart showing a method 1100 for generating a unique identifier associated with a modification of a model, according to some embodiments of the present disclosure. The method 1100 may include, in block 1102, obtaining data representing one or more modifications made to a first version of a model to generate a second version of the model. For example, the system(s) 102 (e.g., the modification component(s) 110) may obtain the model data 112 representing the modification(s) made to the first version of the model 104 to generate the second version of the model 104. As described herein, the modification(s) may be performed by one or more users using one or more computing devices 108.Furthermore, the modification(s) may include fine-tuning the Model 104 to perform a specific task by optimizing the Model 104 to run using specific hardware components (e.g., processors, memory, etc.), by tuning the Model 104 with hyperparameters, by quantizing the Model 104, and / or by performing any other type of model modification technique.

[0087] Procedure 1100 in block B1104 can include generating information associated with at least one or more modifications, based at least on the data. For example, the system(s) 102 (e.g., the information component(s) 114) can use the model data 112 to generate the structured information 116. As described herein, the structured information 116 can contain details describing at least the modification(s) to the first version of the model 104 to generate the second version of the model 104.The structured information 116 may, for example, include information that describes a summary of the changes made, information that describes details of the modification(s) to Model 104, information that describes performance improvements associated with Model 104, information that describes the date on which the modification(s) was / were made, information that describes the user(s) who made / made the modification(s), and / or information that describes any other features and / or modifications associated with Model 104, but is not limited to these.

[0088] Procedure 1100 in block B1106 can include generating a unique identifier associated with the second version of the model, based at least on the information. For example, the system(s) 102 (e.g., the encoding component(s) 118 and / or the identification component(s) 122) can generate the unique identifier 124 using at least the structured information 116. As described herein, the system(s) 102 can generate the unique identifier 124 by first encoding the structured information 116 to generate coded information 120, and then computing the unique identifier 124 using one or more hash algorithms. In the examples, the system(s) 102 can first combine the coded information 120 with a model configuration 128 before calculating the unique identifier 124 using the hash algorithm(s).In some examples, the unique identifier may contain a checksum. In other examples, however, the unique identifier may contain any other type of identifier.

[0089] Procedure 1100 may include, but are not limited to, performing one or more operations using the unique identifier in block B1108. For example, system(s) 102 may perform the operation(s) using the unique identifier 124. As described herein, the operation(s) may include storing the unique identifier 124 in association with the second version of model 104, making the unique identifier 124 available to external repositories 130, decoding the unique identifier 124 to provide the structured information 116 to one or more users, verifying the integrity of the second version of model 104, and / or any other operation.

[0090] Fig. Figure 12 illustrates a flowchart showing a method 1200 for using a unique identifier to provide information associated with a model, according to some embodiments of the present disclosure. The method 1200 may include, in block B1202, receiving a request associated with a model. For example, the system(s) 102 may receive the request associated with the model 104 from the computing device 108. In some examples, the request may be for accessing and / or using the model 104, retrieving information associated with the model 104, retrieving information associated with multiple versions of the model 104, decoding a unique identifier 124 associated with the model 104, and / or any other type of request.

[0091] Procedure 1200 can include in block B1204 the identification of a unique identifier associated with the model. For example, system(s) 102 can identify the unique identifier 124 associated with model 104 from database(s) 126. In some examples, system(s) 102 can also identify one or more additional unique identifiers 124 associated with one or more additional versions associated with model 104.

[0092] Procedure 1200 in block B1206 may include determining information associated with one or more modifications of the model, at least based on the unique identifier. For example, system(s) 102 may use the unique identifier 124 to determine the structured information 116 associated with model 104. For example, system(s) 102 may determine the structured information 116 by decoding the unique identifier 124. In some examples, system(s) may additionally use the additional unique identifier(s) 124 to determine additional structured information 116 associated with the additional version(s) of model 104.

[0093] Procedure 1200 may include in block B1208 the provision of the information associated with one or more modifications. For example, system(s) 102 may provide the structured information 116 associated with model 104. In some examples, system(s) 102 may additionally provide the structured information 116 associated with the additional version(s) of model 104. In some examples, providing the structured information 116 may include sending the structured information 116 to the computing device 108 for display by the computing device 108.

[0094] Fig. Figure 13 illustrates a flowchart showing a method 1300 for using a unique identifier to verify a model, according to some embodiments of the present disclosure. The method 1300 may include, in block B1302, receiving a model and a unique identifier associated with the model. For example, the computing device 108 (and / or an external repository 130) may receive the model 104 and the unique identifier 124 associated with the model 104. As described herein, in some examples, the unique identifier 124 may contain a checksum generated using the structured information 116 associated with the model 104 and one or more hash algorithms. However, in other examples, the unique identifier 124 may contain any other type of identifier that can be used to verify the integrity of the model 104.

[0095] Procedure 1300 in block B1304 may include comparing the unique identifier with an additional unique identifier associated with the model. For example, computing device 108 (and / or external repository 130) may compare the received unique identifier 124 with the additional unique identifier 124. In some examples, the additional unique identifier 124 may be stored by system(s) 102. In other examples, the additional unique identifier 124 may be generated by computing device 108, as by using a similar process to that used by system(s) 102 to generate the received unique identifier 124.

[0096] Procedure 1300 can include in block B1306 determining, at least based on comparison, whether the integrity of the model has been verified. For example, the computing device 108 (and / or the external repository 130) can determine whether the integrity of model 104 is verified, at least based on comparison. For example, the computing device 108 can determine that the integrity of model 104 is verified if the unique identifiers 124 match, or determine that the integrity of model 104 is not verified if the unique identifiers 124 do not match.

[0097] Fig. Figure 14 illustrates a flowchart showing a method 1400 for updating a model map with information describing a modification of the model, according to some embodiments of the present disclosure. The method 1400 may, in block B1402, include determining that one or more modifications were made to a first version of a model to generate a second version of the model. For example, the system(s) 102 (e.g., the detection component(s) 702) may determine that the modification(s) were made to the first version of the model 104 to generate the second version of the model 104. As described herein, in some examples, the system(s) 102 may determine that the modification(s) were made using the unique identifier 124 associated with the second version of the model 104.In some examples, System(s) 102 can determine that the modification(s) was / were made using a different technique, such as System(s) 102 providing the first version of Model 104 for modification, System(s) 102 receiving the second version of Model 104, System(s) receiving a notification that the modification(s) was / were made, and / or any other technique.

[0098] Procedure 1400 in block B1404 can include the determination of information relating to one or more modifications. For example, system(s) 102 can determine the additional information relating to the modification(s). In some examples, system(s) 102 (e.g., decoding component(s) 704) can determine at least some of the additional information, such as the modification information 706, at least based on the decoding of the unique identifier 124. In some examples, system(s) 102 (e.g., testing component(s) 708) can determine at least some of the additional information, such as the performance information 710, at least based on testing the second version of model 104. In other examples, the system(s) 102 may use any other technique to determine the additional information relating to the modification(s).

[0099] Procedure 1400 may include in block B1406 obtaining a model card associated with the first version of the model. For example, system(s) 102 (e.g., update component(s) 712) may obtain model card 714, which is associated with the first version of model 104. In some examples, system(s) 102 may obtain model card 714 using an identifier associated with the first version of model 104 and / or an identifier associated with model card 714. In other examples, however, system(s) 102 may use any other technique to obtain model card 714.

[0100] Procedure 1400 can include updating the model card in block B1408 to contain at least some of the information needed to generate an updated model card. For example, system(s) 102 (e.g., update component(s) 712) can use at least some of the additional information to update model card 714 and generate the updated model card 718. In some examples, system(s) 102 can update model card 714 by adding text to model card 714 that corresponds to at least some of the additional information. In some examples, this process can be repeated so that the updated model card 718 is updated with additional information about one or more additional modifications to model 104.

[0101] Procedure 1400 may, in block B1408, include determining, at least on the basis of the updated model map and one or more capabilities associated with a computing device, that the second version of the model is provided to the computing device. For example, System(s) 102 may receive information specifying one or more capabilities associated with Computing Device 108. As described herein, the capabilities may include, but are not limited to, processing capabilities, storage capabilities, networking capabilities, and / or any other type of capability. System(s) 102 may then use the updated model map 718 to determine that Computing Device 108 is capable of executing the second version of Model 104 based on the one or more capabilities.For example, the system(s) 102 can determine that one or more capabilities meet one or more minimum capabilities for executing the second version of model 104.

[0102] Procedure 1400 may include in block B1410 the sending of data to the computing device for executing the second version of the model. For example, system(s) 102 may send the data for executing the second version of model 104 to computing device 108. In some examples, system(s) 102 sends the data at least on the basis of determining that computing device 108 is capable of executing the second version of model 104.

[0103] Fig. Figure 15 illustrates a flowchart showing another method 1500 for updating a model map with information describing a modification of the model, according to some embodiments of the present disclosure. Method 1500 may, in block B1502, include determining information relating to one or more modifications to a model. For example, the system(s) 102 (e.g., the detection component(s) 702) may determine that the model 104 has been modified to produce a new version of the model 104. The system(s) 102 may then determine the additional information relating to the modification(s). In some examples, the system(s) 102 (e.g., the decoding component(s) 704) may determine at least some of the additional information, such as the modification information 706, at least based on the decoding of the unique identifier 124.In some examples, the system(s) 102 (e.g., the test component(s) 708) can determine at least some of the additional information, such as the performance information 710, at least based on testing the second version of the model 104. In other examples, the system(s) 102 can use any other technique to determine the additional information relating to the modification(s).

[0104] Procedure 1500 can include updating a model card in block B1504 to contain at least some of the information needed to generate an updated model card. For example, system(s) 102 (e.g., update component(s) 712) can use at least some of the additional information for updating model card 714 to generate the updated model card 718. In some examples, system(s) 102 can update model card 714 by adding text to model card 714 that corresponds to at least some of the additional information. In some examples, this process can be repeated so that the updated model card 718 is updated with supplementary additional information about one or more additional modifications to model 104.

[0105] Procedure 1500 can include, in block B1506, saving the updated model map in association with the modified model. For example, system(s) 102 can save the updated model map 716 in association with the new version of model 104. In some examples, system(s) 102 can also synchronize the new version of model 104 and / or the updated model map 716 with one or more external repositories 130, for example, by sending the updated model map 716 and / or the new version of model 104 to the one or more external repositories 130. EXAMPLE CALCULATION DEVICE

[0106] Fig. Figure 16 is a block diagram of an exemplary computing device 1600 suitable for use in implementing at least some embodiments of the present disclosure. The computing device 1600 may include a connection system 1602 that directly or indirectly couples the following devices: main memory 1604, one or more central processing units (CPUs) 1606, one or more graphics processing units (GPUs) 1608, a communication interface 1610, input / output (I / O) ports 1612, input / output components 1614, a power supply 1616, one or more presentation components 1618 (e.g., display(s)), and one or more logic units 1620. In at least one embodiment, the computing device 1600 may include one or more virtual machines (VMs), and / or each of its components may include virtual components (e.g., virtual hardware components).As non-restrictive examples, one or more of the GPUs 1608 can comprise one or more vGPUs, one or more of the CPUs 1606 can comprise one or more vCPUs, and / or one or more of the logic units 1620 can comprise one or more virtual logic units. Thus, a computing device 1600 can contain discrete components (e.g., a complete GPU allocated to the computing device 1600), virtual components (e.g., a portion of a GPU allocated to the computing device 1600), or a combination thereof.

[0107] Although the various blocks of Fig. Where components 16 are shown connected via the connection system 1602, this is not intended as a limitation and is for clarity only. In some embodiments, for example, a presentation component 1618, such as a display device, may be considered an I / O component 1614 (e.g., if the display is a touchscreen). As another example, the CPUs 1606 and / or GPUs 1608 may contain main memory (e.g., the main memory 1604 may represent a storage device in addition to the main memory of the GPUs 1608, the CPUs 1606, and / or other components). Thus, the computing device of Fig. 16 is for illustrative purposes only. No distinction is made between categories such as "workstation", "server", "laptop", "desktop", "tablet", "client device", "mobile device", "handheld device", "game console", "electronic control unit (ECU)", "virtual reality system" and / or other device or system types, since all are within the scope of protection of the computing device of Fig. 16 are being considered.

[0108] The 1602 interconnection system can represent one or more connections or buses, such as an address bus, a data bus, a control bus, or a combination thereof. The 1602 interconnection system can include one or more bus or connection types, such as an Industry Standard Architecture (ISA) bus, an Extended ISA bus, a Video Electronics Standards Association (VESA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI Express (PCIe) bus, and / or another type of bus or connection. In some embodiments, there are direct connections between components. For example, the 1606 CPU can be directly connected to the 1604 main memory. Furthermore, the 1606 CPU can be directly connected to the 1608 GPU.In a direct or point-to-point connection between components, the 1602 connection system can include a PCIe link to establish the connection. In these examples, a PCI bus does not need to be included in the 1600 computing device.

[0109] The 1604 main memory can contain any of a variety of computer-readable media. Computer-readable media can be any available media that the 1600 computing device can access. Computer-readable media can include both volatile and non-volatile media, as well as removable and non-removable media. For example, and without limitation, computer-readable media can include computer storage media and communication media.

[0110] Computer storage media can include both volatile and non-volatile media, and / or removable and non-removable media, implemented in any method or technology for storing information, such as computer-readable instructions, data structures, program modules, and / or other types of data. For example, main memory can store 1604 computer-readable instructions (e.g., representing a program and / or program element, such as an operating system).Computer storage media may, but are not limited to, include RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, Digital Versatile Discs (DVD) or other optical disk storage, magnetic cartridges, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and that the Computing Device 1600 can access. As used herein, computer storage media do not per se include signals.

[0111] Computer storage media can embody computer-readable instructions, data structures, program modules, and / or other types of data in a modulated data signal, such as a carrier wave or other transport mechanism, and may include any media for transmitting information. The term "modulated data signal" can refer to a signal in which one or more of its properties are set or modified to encode information within the signal. Computer storage media may include, but are not limited to, wired media, such as a wired network or a direct-wired connection, and wireless media, such as acoustic, RF, infrared, and other wireless media. Combinations of the foregoing should also be included in the scope of protection of the computer-readable media.

[0112] The CPU(s) 1606 can be configured to execute at least some of the computer-readable instructions to control one or more components of the Computing Device 1600 to perform one or more of the procedures and / or processes described herein. The CPU(s) 1606 can each contain one or more cores (e.g., one, two, four, eight, twenty-eight, seventy-two, etc.) capable of processing a multitude of software threads simultaneously. The CPU(s) 1606 can contain any type of processor and may contain different types of processors depending on the type of Computing Device 1600 implemented (e.g., processors with fewer cores for mobile devices and processors with more cores for servers).Depending on the type of computing device 1600, the processor can be, for example, an Advanced RISC Machine (ARM) processor implemented using Reduced Instruction Set Computing (RISC), or an x86 processor implemented with Complex Instruction Set Computing (CISC). The computing device 1600 can contain one or more CPUs 1606, in addition to one or more microprocessors or additional coprocessors, such as mathematical coprocessors.

[0113] In addition to or as an alternative to the CPU(s) 1606, the CPU(s) 1608 may be configured to execute at least some of the computer-readable instructions to control one or more components of the computing device 1600 to perform one or more of the procedures and / or processes described herein. One or more of the GPUs 1608 may be an integrated GPU (e.g., with one or more of the CPU(s) 1606) and / or one or more of the GPUs 1608 may be a discrete GPU. In embodiments, one or more of the GPUs 1608 may be a coprocessor of one or more of the CPU(s) 1606. The GPU(s) 1608 may be used by the computing device 1600 to render graphics (e.g., 3D graphics) or to perform general-purpose calculations. The GPU(s) 1608 can be used, for example, for general-purpose computing on GPUs (GPGPU).The GPU(s) 1608 can contain hundreds or thousands of cores capable of processing hundreds or thousands of software threads simultaneously. The GPU(s) 1608 can generate pixel data for output images in response to rendering commands (e.g., rendering commands from the CPU(s) 1606 received via a host interface). The GPU(s) 1608 can include graphics memory, such as display memory, for storing pixel data or other suitable data, such as GPGPU data. The display memory can be included as part of the main memory 1604. The GPU(s) 1608 can contain two or more GPUs operating in parallel (e.g., via a link). The link can connect the GPUs directly (e.g., using NVLINK) or connect them via a switch (e.g., using NVSwitch).When combined, each GPU can generate 1608 pixel data or GPGPU data for different sections of an output or for different outputs (e.g., a first GPU for a first image and a second GPU for a second image). Each GPU can have its own dedicated memory or share memory with other GPUs.

[0114] In addition to or as an alternative to the CPU(s) 1606 and / or the GPU(s) 1608, the logic unit(s) 1620 may be configured to execute at least some of the computer-readable instructions to control one or more components of the computing device 1600 to perform one or more of the methods and / or processes described herein. In embodiments, the CPU(s) 1606, the GPU(s) 1608, and / or the logic unit(s) 1620 may discretely or jointly perform any combination of the methods, processes, and / or sections thereof. One or more of the logic units 1620 may be part of and / or integrated within one or more of the CPU(s) 1606 and / or the GPU(s) 1608, and / or one or more of the logic units 1620 may be discrete components or otherwise separate from the CPU(s) 1606 and / or the GPU(s) 1608.In embodiments, one or more of the logic units 1620 can be a co-processor of one or more of the CPUs 1606 and / or one or more of the GPUs 1608.

[0115] Examples of Logic Unit(s) 1620 include one or more processing cores and / or components thereof, such as Data Processing Units (DPUs), Tensor Cores (TCs), Tensor Processing Units (TPUs), Pixel Visual Cores (PVCs), Vision Processing Units (VPUs), Graphics Processing Clusters (GPCs), Texture Processing Clusters (TPCs), Streaming Multiprocessors (SMs), Tree Traversal Units (TTUs), Artificial Intelligence Accelerators (AIAs), Deep Learning Accelerators (DLAs), and Programmable Vision Accelerators (PVAs) – which include one or more Direct Access Memory (DIM) units. Memory Access (DMA) systems,one or more vision or vector processing units (VPUs), one or more pixel processing engines (PPEs) - e.g. B. including a 2D array of processing elements, each communicating north, south, east and west with one or more other processing elements in the array, one or more decoupled accelerators or units (e.g., decoupled lookup table (DLUT) accelerators or units), etc., vision processing units (VPUs), optical flow accelerators (OFAs), field programmable gate arrays (FPGAs), neuromorphic chips, quantum processing units (QPUs), associative process units (APUs), arithmetic logic units (ALUs),May contain application-specific integrated circuits (ASICs), floating-point units (FPUs), input / output (I / O) elements, peripheral component interconnect (PCI) or PCI Express (PCIe) elements, and / or the like.

[0116] The 1610 communication interface can include one or more receivers, transmitters, and / or transceivers that enable the 1600 computing device to communicate with other computers over an electronic network, including wired and / or wireless communication. The 1610 communication interface can include components and functions that enable communication over a variety of different networks, such as wireless networks (e.g., Wi-Fi, Z-Wave, Bluetooth, Bluetooth LE, ZigBee, etc.), wired networks (e.g., Ethernet or InfiniBand communication), low-power wide-area networks (e.g., LoRaWAN, SigFox, etc.), and / or the Internet.In one or more embodiments, the logic unit(s) 1620 and / or the communication interface 1610 may contain one or more data processing units (DPUs) to transfer data received via a network and / or the connection system 1602 directly to one or more GPUs 1608 (e.g., a memory thereof).

[0117] The I / O ports 1612 enable the computing device 1600 to be logically coupled with other devices, including the I / O components 1614, the presentation component(s) 1618, and / or other components, some of which may be built into (e.g., integrated with) the computing device 1600. Illustrative I / O components 1614 include a microphone, mouse, keyboard, joystick, gamepad, game controller, satellite dish, scanner, printer, wireless device, etc. The I / O components 1614 can provide a natural user interface (NUI) that processes air gestures, speech, or other physiological inputs generated by a user. In some cases, the inputs can be transmitted to a suitable network element for further processing.A NUI can implement any combination of speech capture, stylus capture, face capture, biometric capture, gesture capture (both on-screen and off-screen), air gestures, head and eye tracking, and touch capture (as described in more detail below) associated with a display of the Computing Device 1600. The Computing Device 1600 can include depth cameras, such as stereoscopic camera systems, infrared camera systems, RGB camera systems, touchscreen technology, and combinations thereof, for gesture capture and recognition. Additionally, the Computing Device 1600 can include accelerometers or gyroscopes (e.g., as part of an inertial measurement unit (IMU)) that enable motion detection. In some examples, the output from the accelerometers or gyroscopes can be used by the Computing Device 1600 to render immersive augmented reality or virtual reality.

[0118] The 1616 power supply can include a hardwired power supply, a battery power supply, or a combination of both. The 1616 power supply can power the 1600 computing device to enable the operation of its components.

[0119] The presentation component(s) 1618 can include a display (e.g., a monitor, touchscreen, television screen, head-up display (HUD), other display types, or a combination thereof), speakers, and / or other presentation components. The presentation component(s) 1618 can receive data from other components (e.g., the GPU(s) 1608, the CPU(s) 1606, DPUs, etc.) and output the data (e.g., as an image, video, sound, etc.). EXEMPLARY DATA CENTER

[0120] Fig. Figure 17 illustrates an exemplary data center 1700 that can be used in at least one embodiment of the present disclosure. The data center 1700 can include an infrastructure layer 1710, a framework layer 1720, a software layer 1730, and / or an application layer 1740.

[0121] As in Fig. As shown in Figure 17, the infrastructure layer 1710 of the data center can contain a resource orchestrator 1712, grouped computer resources 1714 and node computer resources (“node RRs”) 1716(1)-1716(N), where “N” is any positive integer. In at least one embodiment, the node RRs 1716(1)-1716(N) may include, but are not limited to, any number of central processing units (CPUs) or other processors (including DPUs, accelerators, field programmable gate arrays (FPGAs), graphics processing units or graphics processing units (GPUs), etc.), memory devices (e.g., dynamic solid memory), storage devices (e.g., solid-state or disk drives), network input / output (NW I / O) devices, network switches, virtual machines (VMs), power supply modules and / or cooling modules, etc.In some embodiments, one or more node RRs among node RRs 1716(1)-1716(N) may correspond to a server that has one or more of the aforementioned computing resources. Furthermore, in some embodiments, node RRs 1716(1)-1716(N) may contain one or more virtual components, such as vGPUs, vCPUs, and / or the like, and / or one or more of node RRs 1716(1)-1716(N) may correspond to a virtual machine (VM).

[0122] In at least one embodiment, the grouped compute resources 1714 can contain separate groupings of node RRs 1716, which are housed in one or more racks (not shown) or in many racks in data centers at different geographic locations (also not shown). Separate groupings of node RRs 1716 within grouped compute resources 1714 can contain grouped compute, network, memory, or storage resources that can be configured or allocated to support one or more workloads. In at least one embodiment, multiple node RRs 1716, including the CPUs, GPUs, DPUs, and / or other processors, can be grouped in one or more racks to provide compute resources for supporting one or more workloads.The one or more racks can also contain any number of power supply modules, cooling modules and / or network switches in any combination.

[0123] The resource orchestrator 1712 can configure or otherwise control one or more node RRs 1716(1)-1716(N) and / or grouped compute resources 1714. In at least one embodiment, the resource orchestrator 1712 can include an entity for managing the software design infrastructure (SDI) for the data center 1700. The resource orchestrator 1712 can include hardware, software, or a combination thereof.

[0124] In at least one embodiment, as in Fig. As shown in Figure 17, the framework layer 1720 can contain a job scheduler 1728, a configuration manager 1734, a resource manager 1736, and / or a distributed file system 1738. The framework layer 1720 can contain a framework that supports the software 1732 of the software layer 1730 and / or an application 1742 of the application layer 1740. The software 1732 or the application 1742 can each contain web-based service software or applications, such as those provided by Amazon Web Services, Google Cloud, and Microsoft Azure. The framework layer 1720 can be a type of free and open-source software web application framework, such as Apache Spark. TM(hereinafter referred to as "Spark"), which can use a distributed file system 1738 for processing large amounts of data (e.g., "Big Data"), without being limited to it. In at least one embodiment, the job scheduler 1728 can include a Spark driver to facilitate the scheduling of workloads supported by different layers of the data center 1700. The configuration manager 1734 can be able to configure different layers, such as the software layer 1730 and the framework layer 1720, which contain Spark and the distributed file system 1738, to support the processing of large amounts of data. The resource manager 1736 can be able to manage clustered or grouped computer resources that are allocated or assigned to support the distributed file system 1738 and the job scheduler 1728.In at least one embodiment, the clustered or grouped computing resources can include the grouped computing resource 1714 on the infrastructure layer 1710 of the data center. The resource manager 1736 can coordinate with the resource orchestrator 1712 to manage these allocated or assigned computing resources.

[0125] In at least one embodiment, the software 1730 contained in software layer 1732 may include software used by at least sections of the node RRs 1716(1)-1716(N), the grouped compute resources 1714, and / or the distributed file system 1738 of framework layer 1720. One or more types of software may include, but are not limited to, web page search software, email virus scanning software, database software, and streaming video content software.

[0126] In at least one embodiment, the application(s) 1742 contained in the application layer 1740 may contain one or more types of applications used by at least sections of the node RRs 1716(1)-1716(N), the grouped compute resources 1714, and / or the distributed file system 1738 of the framework layer 1720. One or more types of applications may include, but are not limited to, any number of genome applications, cognitive computations, and machine learning applications, including training or inference software, machine learning framework software (e.g., PyTorch, TensorFlow, Caffe, etc.), and / or other machine learning applications used in conjunction with one or more embodiments.

[0127] In at least one embodiment, a configuration manager 1734, a resource manager 1736, and a resource orchestrator 1712 can implement any number and type of self-modifying actions based on any set and type of data acquired in any technically feasible manner. Self-modifying actions can relieve a data center operator of data center 1700 of potentially making poor configuration decisions and potentially avoiding underutilized and / or poorly functioning sections of a data center.

[0128] The 1700 data center may contain tools, services, software, or other resources to train one or more machine learning models or to predict or infer information using one or more machine learning models according to one or more embodiments described herein. For example, a machine learning model(s) may be trained by calculating weighting parameters according to a neural network architecture, using software and / or computing resources described above with reference to the 1700 data center.In at least one embodiment, trained or deployed machine learning models corresponding to one or more neural networks can be used to infer or predict information using the resources described above with reference to the Computing Center 1700 by using weighting parameters calculated by one or more training techniques such as, but not limited to, those described herein.

[0129] In at least one embodiment, the data center can use 1700 CPUs, application-specific integrated circuits (ASICs), GPUs, FPGAs, and / or other hardware (or equivalent virtual computing resources) to perform training and / or inference using the resources described above. Furthermore, one or more of the software and / or hardware resources described above can be configured as a service to allow users to train or infer information, such as image capture, speech capture, or other artificial intelligence services. EXEMPLARY NETWORK ENVIRONMENTS

[0130] Network environments suitable for implementing embodiments of the disclosure may include one or more client devices, servers, network-attached storage (NAS), other backend devices, and / or other device types. The client devices, servers, and / or other device types (e.g., each device) may run on one or more instances of the computing device(s) 1600. Fig. 16 are implemented - e.g., each device may contain similar components, features, and / or functionality to the computing device(s) 1600. If backend devices (e.g., servers, NAS, etc.) are implemented, the backend devices may also be included as part of a data center 1700, an example of which is given herein with reference to Fig. 17 is described in more detail.

[0131] The components of a network environment can communicate with each other over a network, which can be wired, wireless, or both. The network can contain multiple networks or a network of networks. For example, the network can contain one or more wide area networks (WANs), one or more local area networks (LANs), one or more public networks such as the internet and / or a public switched telephone network (PSTN), and / or one or more private networks. If the network includes a wireless telecommunications network, components such as a base station, a communications tower, or even access points (as well as other components) can provide wireless connectivity.

[0132] Compatible network environments can include one or more peer-to-peer network environments—in which case a server cannot be included in a network environment—and one or more client-server network environments—in which case one or more servers can be included in a network environment. In peer-to-peer network environments, the functionality described herein can be implemented on any number of client devices with reference to a server.

[0133] In at least one embodiment, a network environment can include one or more cloud-based network environments, a distributed computing environment, a combination thereof, etc. A cloud-based network environment can include a framework layer, a job scheduler, a resource manager, and a distributed file system implemented on one or more servers, which may include one or more core network servers and / or edge servers. A framework layer can include a framework for supporting software of a software layer and / or one or more applications of an application layer. The software or application(s) can each include web-based service software or applications. In embodiments, one or more of the client devices can use the web-based service software or applications (e.g.,by accessing the service software and / or applications via one or more application programming interfaces (APIs). The framework layer can be a type of free and open-source software web application framework, such as one that uses a distributed file system for processing large amounts of data (e.g., "Big Data"), but is not limited to that.

[0134] A cloud-based network environment can provide cloud computing and / or cloud storage, performing any combination (or parts thereof) of the computing and / or data storage functions described herein. Each of these different functions can be distributed across multiple locations of central or core servers (e.g., one or more data centers, which may be distributed across a state, region, country, the globe, etc.). If a connection to a user (e.g., a client device) is relatively close to an edge server, the core server(s) may offload at least some functionality to the edge server(s). A cloud-based network environment can be private (e.g., restricted to a single organization), public (e.g., available to many organizations), and / or a combination thereof (e.g., a hybrid cloud environment).

[0135] The client device(s) may include at least some of the components, features, and functions described herein with respect to Fig.The 16 exemplary computing device(s) described in Section 1600 may include 1600. As an example, and not as a limitation, a client device may be a personal computer (PC), a laptop, a mobile device, a smartphone, a tablet computer, a smartwatch, a portable computer, a personal digital assistant (PDA), an MP3 player, a virtual reality headset, a global positioning system (GPS) or global positioning device, a video player, a video camera, a surveillance device or surveillance system, a vehicle, a hydrofoil, a virtual machine, a drone, a robot, a handheld communication device, a hospital device, a gaming device or gaming system, an entertainment system, a vehicle computer system, an embedded system controller, a remote control, a device, a consumer electronics device, a workstation, an edge device,any combination of these described devices or any other suitable device may be embodied.

[0136] The revelation can be described in the general context of computer code or machine-usable instructions, including computer-executable instructions such as program modules that are executed by a computer or other machine, such as a personal data assistant or other handheld device. Generally, program modules contain routines, programs, objects, components, data structures, etc., and refer to code that performs specific tasks or implements certain abstract types of data. The revelation can be practiced in a variety of system configurations, including handheld devices, consumer electronics, general-purpose computers, more specialized computing devices, etc. The revelation can also be practiced in distributed computing environments, where tasks are performed by remote processing devices connected to each other via a network for communication.

[0137] As used herein, any mention of "and / or" in relation to two or more elements should be interpreted as referring to only one element or a combination of elements. For example, "element A, element B and / or element C" may include only element A, only element B, only element C, element A and element B, element A and element C, element B and element C, or elements A, B, and C. Furthermore, "at least one of element A or element B" may include at least one of element A, at least one of element B, or at least one of element A and at least one of element B. Additionally, "at least one of element A and element B" may include at least one of element A, at least one of element B, or at least one of element A and at least one of element B.

[0138] The subject matter of this disclosure is specifically described herein to satisfy legal requirements. However, the description itself is not intended to limit the scope of protection afforded by this disclosure. Rather, the inventors have considered that the claimed subject matter may also be embodied in other ways to include various steps or combinations of steps similar to those described in this document, in conjunction with other present or future technologies. Although the terms "step" and / or "block" may be used herein to denote various elements of the methods employed, these terms should not be interpreted as implying a specific sequence among or between the various steps disclosed herein, except where the sequence of each step is expressly described. EXAMPLE PARAGRAPHS A: A method comprising: obtaining data representing one or more modifications made to a first version of a model in order to generate a second version of the model; generating a checksum for the second version of the model based at least on the data representing the one or more modifications; and providing the second version of the model together with the checksum to one or more computing devices, wherein the second version of the model can be verified using the checksum. B: Method according to paragraph A, wherein the data representing the one or more modifications represent at least one of the following: an architecture associated with the second version of the model; a dataset used to train the second version of the model; a risk score associated with the second version of the model; a license associated with the second version of the model; a performance score associated with the second version of the model; or one or more types of hardware for which the second version of the model was optimized. C: Method according to paragraph A or paragraph B, wherein generating the checksum for the second version of the model comprises: generating information describing the one or more modifications, at least on the basis of the data representing the one or more modifications; and determining the checksum associated with the second version of the model, at least on the basis of the information. D: Method according to paragraph C, wherein determining the checksum associated with the second version of the model, based on the information, comprises at least the following: encoding the information to generate coded information; and applying one or more algorithms to the coded information to obtain the checksum associated with the second version of the model. E: Method according to paragraph C or D, wherein the information describing the one or more modifications includes at least one of the following: a first text describing the one or more modifications; a second text describing a data set used to perform the one or more modifications; a third text describing a modification of the performance associated with the second version of the model; a fourth text describing a time at which the one or more modifications were made; a fifth text describing one or more users who performed the one or more modifications; or a sixth text describing the hardware that performed the one or more modifications. F: Procedure according to one of the paragraphs AE, which further comprises: determining information describing the one or more modifications, at least on the basis of the checksum; and providing the information to the one or more computing devices. G: Method according to any of paragraphs AF, further comprising: providing the first version of the model to one or more second computing devices, wherein the obtaining of the data representing the one or more modifications is based on at least the first version of the model modified using the one or more second computing devices. H: Procedure according to one of the paragraphs AG, which further comprises: obtaining second data representing one or more second modifications made to the first version of the model or the second version of the model in order to generate a third version of the model; generating a second checksum associated with the third version of the model, at least on the basis of the second data representing the one or more second modifications; and storing the second checksum in association with the third version of the model. I: Procedure according to any of paragraphs AH, wherein the one or more modifications are assigned to at least one of the following: fine-tuning the first version of the model to generate the second version of the model; optimizing the first version of the model for one or more hardware components; or performing quantization on the first version of the model to generate the second version of the model. J: System comprising: one or more processors for the following: providing a first version of a model to one or more computing devices; at least on the basis of the first version of the model, modified using the one or more computing devices, receiving data representing one or more modifications made to generate a second version of the model; at least on the basis of the data, generating a unique identifier for the second version of the model; and storing the unique identifier in association with the second version of the model. K: System according to paragraph J, wherein the one or more processors further serve to: provide the unique identifier together with the second version of the model to one or more second computing devices, wherein the second version of the model can be verified using the unique identifier. L: System according to either paragraph J or paragraph K, wherein the one or more processors further serve to: generate at least on the basis of the data information describing the one or more modifications, wherein the unique identifier is generated at least on the basis of the information. M: System according to paragraph L, wherein the generation of the unique identifier for the second version of the model comprises: encoding the information to generate encoded information; and generating the unique identifier associated with the second version of the model, at least on the basis of hashing the encoded information using one or more algorithms. N: System according to one of paragraphs JM, wherein the one or more processors further serve to: determine, at least on the basis of the unique identifier, information describing the one or more modifications; and provide the information together with the second version of the model to the one or more second computing devices. O: System according to one of paragraphs JN, wherein the one or more processors further serve to: provide the second version of the model to one or more second computing devices; at least on the basis of the second version of the model, modified using the one or more second computing devices, obtain data representing one or more second modifications made to generate a third version of the model; at least on the basis of the data, generate a second unique identifier associated with the third version of the model; and store the second unique identifier in association with the third version of the model. P: System according to paragraph O, wherein the one or more processors further serve to: provide one or more computing devices with, using the unique identifier of the first, information describing the one or more modifications; provide the one or more computing devices with, using the second unique identifier of, the second, information describing the one or more second modifications; receive from the one or more computing devices a request for the second version of the model; and provide the one or more computing devices with, at least on the basis of the request, the second version of the model. Q: System according to any of paragraphs JP, wherein the system consists of at least one of the following: a control system for an autonomous or semi-autonomous machine; a perception system for an autonomous or semi-autonomous machine; a system for performing one or more simulation operations; a system for performing one or more digital twin operations; a system for performing light transport simulations; a system for performing collaborative content creation for 3D assets; a system that provides one or more cloud gaming applications; a system for performing one or more deep learning operations; a system implemented using an edge device; a system implemented using a robot; a system for performing one or more generative AI operations;a system for performing one or more operations using one or more small language models; a system for performing operations using one or more large language models (LLMs); a system for performing operations using one or more vision language models (VLMs); a system for performing operations using one or more multimodal language models; a system for performing one or more operations using conversational AI; a system for generating synthetic data; a system for presenting at least one type of virtual reality content, augmented reality content, or mixed reality content; systems implementing one or more multimodal language models; systems using or employing one or more inference microservices;Systems that include one or more machine learning models in a service or microservice together with an OS-level virtualization package (e.g., a container); a system that contains one or more virtual machines (VMs); a system that is at least partially deployed in a data center; or a system that is at least partially deployed using cloud computing resources. R: One or more processors comprising: processing circuitry for the following: generating information describing one or more modifications made to a first version of a model to generate a second version of the model; generating a unique identifier for the second version of the model based at least on the information; and storing the unique identifier in association with the second version of the model. S: One or more processors according to paragraph R, wherein the unique identifier is generated by: encoding the information to produce encoded information; and applying one or more algorithms to the encoded information to obtain the unique identifier for the second version of the model. T: One or more processors according to either paragraph R or paragraph S, wherein the one or more processors are / are included in at least one of the following: a control system for an autonomous or semi-autonomous machine; a perception system for an autonomous or semi-autonomous machine; a system for performing one or more simulation operations; a system for performing one or more digital twin operations; a system for performing light transport simulations; a system for performing collaborative content creation for 3D assets; a system that provides one or more cloud gaming applications; a system for performing one or more deep learning operations; a system implemented using an edge device; a system implemented using a robot; a system for performing one or more generative AI operations;a system for performing one or more operations using one or more small language models; a system for performing operations using one or more large language models (LLMs); a system for performing operations using one or more vision language models (VLMs); a system for performing operations using one or more multimodal language models; a system for performing one or more operations using conversational AI; a system for generating synthetic data; a system for presenting at least one type of virtual reality content, augmented reality content, or mixed reality content; systems implementing one or more multimodal language models; systems using or employing one or more inference microservices;Systems that include one or more machine learning models in a service or microservice along with an OS-level virtualization package (e.g., a container); a system that contains one or more virtual machines (VMs); a system that is at least partially deployed in a data center; or a system that is at least partially deployed using cloud computing resources. U: A procedure comprising: determining that one or more modifications have been made to a first version of a model to generate a second version of the model; obtaining a model map associated with the first version of the model; updating the model map to include at least some of the information relating to one or more modifications in order to generate an updated model map; determining, based at least on the updated model map and one or more capabilities associated with a computing device, to provide the second version of the model to the computing device; and transferring data to the computing device to execute the second version of the model. V: Procedure according to paragraph U, which further comprises: obtaining a unique identifier associated with the second version of the model, wherein the determination that one or more modifications were made to the first version of the model is based at least on the unique identifier. W: Procedure according to either paragraph U or paragraph V, wherein the information relating to the one or more modifications is determined by: obtaining a unique identifier associated with the second version of the model, processing the unique identifier to determine the information relating to the one or more modifications. X: Procedure according to one of the paragraphs UW, wherein the information relating to the one or more modifications is determined by: performing one or more performance tests associated with the second version of the model; and identifying the information relating to the one or more modifications, at least on the basis of the one or more performance tests. Y: A procedure according to one of paragraphs UX, wherein the information relating to the one or more modifications includes at least one of the following: first information specifying a type of modification made to the first version of the model; second information specifying a time at which the one or more modifications were made; third information specifying one or more hardware components for which the second version of the model was optimized; fourth information specifying one or more performance characteristics associated with the execution of the second version of the model using the one or more hardware components; fifth information specifying one or more performance characteristics associated with the second version of the model; or a unique identifier associated with the second version of the model. Z: Procedure according to any of paragraphs UY, further comprising: obtaining second information relating to one or more second modifications made to the second version of the model to produce a third version of the model; and updating the updated model map to include at least part of the second information. AA: Procedure according to one of the paragraphs UZ, which further comprises at least one of the following: storing the updated model map associated with the second version of the model; or providing the second version of the model and / or the updated model map to one or more systems. AB: System comprising one or more processors for the following: determining information relating to one or more modifications made to a first version of a model in order to generate a second version of the model; accessing a model map associated with the first version of the model; updating the model map to contain at least some of the information to generate an updated model map; and storing the updated model map in association with the second version of the model. AC: System according to paragraph AB, wherein the one or more processors further serve to: determine, at least on the basis of the updated model map and one or more capabilities associated with a computing device, that the second version of the model is provided to the computing device; and send data to the computing device to execute the second version of the model. AD: System according to either paragraph AB or paragraph AC, wherein the one or more processors further serve to: obtain a unique identifier that is assigned to the second version of the model; and at least on the basis of the unique identifier, determine that the one or more modifications have been made to the first version of the model, updating the model map at least on the basis of the determination that the one or more modifications have been made. AE: System according to one of paragraphs AB-AD, wherein the information relating to the one or more modifications is determined by: obtaining a unique identifier associated with the second version of the model, processing the unique identifier to determine the information relating to the one or more modifications. AF: System according to one of paragraphs AB-AE, wherein the information relating to the one or more modifications is determined by: performing one or more performance tests associated with the second version of the model; and determining the information relating to the one or more modifications, at least on the basis of the one or more performance tests. AG: System according to one of paragraphs AB-AF, wherein the information relating to the one or more modifications is determined by: obtaining data associated with carrying out the one or more modifications to generate the second version of the model; and determining the information relating to the one or more modifications, at least on the basis of the data. AH: System according to one of paragraphs AB-AG, wherein the information relating to the one or more modifications includes at least one of the following: first information specifying a type of modification made to the first version of the model; second information specifying a time at which the one or more modifications were made; third information specifying one or more hardware components for which the second version of the model was optimized; fourth information specifying one or more performance characteristics associated with the execution of the second version of the model using the one or more hardware components; fifth information specifying one or more performance characteristics associated with the second version of the model; or a unique identifier associated with the second version of the model. AI: System according to any of paragraphs AB-AH, wherein the one or more processors further serve to: determine second information relating to one or more second modifications made to the first version of the model in order to generate a third version of the model; and update the updated model map to include at least some of the second information. AJ: System according to paragraph AI, wherein: one or more modifications are associated with the optimization of the second version of the model using a first type of hardware; at least one part of the information indicates one or more first performance characteristics associated with the execution of the second version of the model on the first type of hardware; one or more second modifications are associated with the optimization of the third version of the model using a second type of hardware; and at least one part of the second information indicates one or more second performance characteristics associated with the execution of the second version of the model on the second type of hardware. AK: System according to one of paragraphs AB-AJ, wherein the system consists of at least one of the following: a control system for an autonomous or semi-autonomous machine; a perception system for an autonomous or semi-autonomous machine; a system for performing one or more simulation operations; a system for performing one or more digital twin operations; a system for performing light transport simulations; a system for performing collaborative content creation for 3D assets; a system that provides one or more cloud gaming applications; a system for performing one or more deep learning operations; a system that is implemented using an edge device; a system that is implemented using a robot; a system for performing one or more generative AI operations;a system for performing one or more operations using one or more small language models; a system for performing operations using one or more large language models (LLMs); a system for performing operations using one or more vision language models (VLMs); a system for performing operations using one or more multimodal language models; a system for performing one or more operations using conversational AI; a system for generating synthetic data; a system for presenting at least one type of virtual reality content, augmented reality content, or mixed reality content; systems implementing one or more multimodal language models; systems using or employing one or more inference microservices;Systems that include one or more machine learning models in a service or microservice along with an OS-level virtualization package (e.g., a container); a system that contains one or more virtual machines (VMs); a system that is at least partially deployed in a data center; or a system that is at least partially deployed using cloud computing resources. AL: One or more processors comprising processing circuitry for: obtaining a unique identifier associated with a version of a model; determining, at least on the basis of the unique identifier, information relating to one or more modifications to an earlier version of the model in order to generate the version of the model; updating a model map associated with the earlier version of the model, using at least some of the information to generate an updated model map; and storing the updated model map in association with the version of the model. AM: One or more processors according to paragraph AL, wherein the processing circuits further serve to: determine, at least on the basis of the updated model map and one or more capabilities associated with a computing device, that the version of the model is provided to the computing device; and send data to the computing device to execute the second version of the model. AN: One or more processors according to either paragraph AL or paragraph AM, wherein the one or more processors are / are included in at least one of the following: a control system for an autonomous or semi-autonomous machine; a perception system for an autonomous or semi-autonomous machine; a system for performing one or more simulation operations; a system for performing one or more digital twin operations; a system for performing light transport simulations; a system for performing collaborative content creation for 3D assets; a system that provides one or more cloud gaming applications; a system for performing one or more deep learning operations; a system that is implemented using an edge device; a system that is implemented using a robot; a system for performing one or more generative AI operations;a system for performing one or more operations using one or more small language models; a system for performing operations using one or more large language models (LLMs); a system for performing operations using one or more vision language models (VLMs); a system for performing operations using one or more multimodal language models; a system for performing one or more operations using conversational AI; a system for generating synthetic data; a system for presenting at least one type of virtual reality content, augmented reality content, or mixed reality content; systems implementing one or more multimodal language models; systems using or employing one or more inference microservices;Systems that include one or more machine learning models in a service or microservice along with an OS-level virtualization package (e.g., a container); a system that contains one or more virtual machines (VMs); a system that is at least partially deployed in a data center; or a system that is at least partially deployed using cloud computing resources.

[0139] It is understood that the aspects and embodiments described above are only exemplary and that changes to details may be made within the scope of protection of the claims.

[0140] Each device, method and feature disclosed in the description and (where applicable) in the claims and drawings may be provided independently or in any suitable combination.

[0141] The reference numerals appearing in the claims are for illustrative purposes only and are not intended to have any limiting effect on the scope of protection of the claims.

Claims

[1] Procedure comprising the following: Determine that one or more modifications were made to a first version of a model to create a second version of the model; Receipt of a model card that is associated with the first version of the model; Updating the model map to include at least some of the information relating to one or more modifications in order to generate an updated model map; Determine, at least on the basis of the updated model map and one or more capabilities assigned to a computing device, to provide the second version of the computing device's model; and Transferring data to the computing device to run the second version of the model. [2] The method of claim 1, further comprising: Obtaining a unique identifier that is assigned to the second version of the model, where determining that one or more modifications were made to the first version of the model is based at least on the unique identifier. [3] Method according to claim 1 or 2, wherein the information relating to the one or more modifications is determined by the following: Obtaining a unique identifier that is assigned to the second version of the model, Processing the unique identifier to determine the information relating to one or more modifications. [4] A method according to any of the preceding claims, wherein the information relating to the one or more modifications is determined by the following: Performing one or more performance tests associated with the second version of the model; and Identifying the information relating to the one or more modifications, at least on the basis of the one or more performance tests. [5] A method according to any of the preceding claims, wherein the information relating to the one or more modifications comprises at least one of the following: initial information indicating a type of modification that was made to the first version of the model; second piece of information indicating the time at which one or more modifications were made; third information specifying one or more hardware components for which the second version of the model was optimized; fourth, information specifying one or more performance characteristics associated with the execution of the second version of the model using the one or more hardware components; fifth piece of information specifying one or more performance characteristics associated with the second version of the model; or a unique identifier that is assigned to the second version of the model. [6] A method according to any of the preceding claims, further comprising: Receiving second-party information relating to one or more second-party modifications made to the second version of the model to produce a third version of the model; and Update the updated model map to include at least some of the second set of information. [7] A method according to any of the preceding claims, further comprising at least one of the following: Saving the updated model map associated with the second version of the model; or Providing the second version of the model and / or the updated model map to one or more systems. [8] System comprising one or more processors for the following: Determining information relating to one or more modifications made to an initial version of a model to create a second version of the model; Access a model card that is associated with the first version of the model; Updating the model map to include at least some of the information needed to generate an updated model map; and Saving the updated model map in association with the second version of the model. [9] System according to claim 8, wherein the one or more processors further serve to: Determine, at least on the basis of the updated model map and one or more capabilities associated with a computing device, that the second version of the model is provided to the computing device; and Sending data to the computing device to execute the second version of the model. [10] System according to claim 8 or 9, wherein the one or more processors further serve to: Obtaining a unique identifier that is assigned to the second version of the model; and Determine, at least on the basis of the unique identifier, that one or more modifications were made to the first version of the model. the model map is updated at least on the basis of the determination that one or more modifications have been made. [11] System according to any one of claims 8 to 10, wherein the information relating to the one or more modifications is determined by the following: Obtaining a unique identifier that is assigned to the second version of the model, Processing the unique identifier to determine the information relating to one or more modifications. [12] System according to any one of claims 8 to 11, wherein the information relating to the one or more modifications is determined by the following: Performing one or more performance tests associated with the second version of the model; and Determining the information relating to one or more modifications, at least on the basis of one or more performance tests. [13] System according to any one of claims 8 to 12, wherein the information relating to one or more modifications is determined by the following: Obtaining data associated with performing one or more modifications to generate the second version of the model; and Determine the information relating to one or more modifications, at least on the basis of the data. [14] System according to any one of claims 8 to 13, wherein the information relating to the one or more modifications comprises at least one of the following: initial information indicating a type of modification that was made to the first version of the model; second piece of information indicating the time at which one or more modifications were made; third information specifying one or more hardware components for which the second version of the model was optimized; fourth, information specifying one or more performance characteristics associated with the execution of the second version of the model using the one or more hardware components; fifth piece of information specifying one or more performance characteristics associated with the second version of the model; or a unique identifier that is assigned to the second version of the model. [15] System according to any one of claims 8 to 14, wherein the one or more processors further serve to: Determining secondary information relating to one or more secondary modifications made to the first version of the model to produce a third version of the model; and Update the updated model map to include at least some of the second set of information. [16] System according to claim 15, wherein: the one or more modifications are associated with the optimization of the second version of the model using a first type of hardware; which at least part of the information indicates one or more initial performance characteristics that are associated with the execution of the second version of the model on the first type of hardware; which are associated with one or more second modifications of the optimization of the third version of the model using a second type of hardware; and which at least part of the second information indicates one or more second performance characteristics that are associated with the execution of the second version of the model on the second type of hardware. [17] System according to any one of claims 8 to 16, wherein the system consists of at least one of the following: a control system for an autonomous or semi-autonomous machine; a perception system for an autonomous or semi-autonomous machine; a system for performing one or more simulation processes; a system for performing one or more digital twin operations; a system for performing light transport simulations; a system for conducting collaborative content creation for 3D assets; a system that provides one or more cloud gaming applications; a system for performing one or more deep learning operations; a system implemented using an edge device; a system that is implemented using a robot; a system for performing one or more operations using generative AI; a system for performing one or more operations using one or more small language models (SLMs); a system for performing operations using one or more large language models (LLMs); a system for performing operations using one or more Visual Language Models (VLMs); a system for performing operations using one or more multimodal language models; a system for performing one or more operations using conversational AI; a system for generating synthetic data; a system for presenting at least one type of virtual reality content, augmented reality content, or mixed reality content; Systems that implement one or more multimodal language models; Systems that use or employ one or more inference microservices; Systems that incorporate one or more machine learning models into a service or microservice along with an OS-level virtualization package (e.g., a container); a system that contains one or more virtual machines (VMs); a system that is at least partially implemented in a data center; or a system that is implemented at least partially using cloud computing resources. [18] One or more processors comprising: processing circuits for the following: Obtaining a unique identifier that is assigned to a version of a model; at least on the basis of the unique identifier, determine information relating to one or more modifications to an earlier version of the model in order to generate the version of the model; Updating a model map associated with the earlier version of the model, using at least some of the information to generate an updated model map; and to save the updated model map in association with the version of the model. [19] One or more processors according to claim 18, wherein the processing circuits further serve to: Determine, at least on the basis of the updated model map and one or more capabilities associated with a computing device, that the version of the model is provided to the computing device; and Sending data to the computing device to execute the second version of the model. [20] One or more processors according to claim 18 or 19, wherein the one or more processors is / are contained in at least one of the following: a control system for an autonomous or semi-autonomous machine; a perception system for an autonomous or semi-autonomous machine; a system for performing one or more simulation processes; a system for performing one or more digital twin operations; a system for performing light transport simulations; a system for conducting collaborative content creation for 3D assets; a system that provides one or more cloud gaming applications; a system for performing one or more deep learning operations; a system implemented using an edge device; a system that is implemented using a robot; a system for performing one or more operations using generative AI; a system for performing one or more operations using one or more small language models (SLMs); a system for performing operations using one or more large language models (LLMs); a system for performing operations using one or more Visual Language Models (VLMs); a system for performing operations using one or more multimodal language models; a system for performing one or more operations using conversational AI; a system for generating synthetic data; a system for presenting at least one type of virtual reality content, augmented reality content, or mixed reality content; Systems that implement one or more multimodal language models; Systems that use or employ one or more inference microservices; Systems that incorporate one or more machine learning models into a service or microservice along with an OS-level virtualization package (e.g., a container); a system that contains one or more virtual machines (VMs); a system that is at least partially implemented in a data center; or a system that is implemented at least partially using cloud computing resources.

Images