Distributed Ledger for Machine Learning Model Versioning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The challenge lies in tracking and managing the versions of machine-learning models deployed in production systems, as updates can lead to different versions being used simultaneously, making it difficult for auditors or regulators to assess their accuracy, bias, or performance.
Innovation Solution
The use of distributed ledgers, such as blockchain, to automatically certify machine-learning models and store this certification, allowing for controlled usage and precise auditing of these models.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If machine-learning models are updated frequently to improve performance, then model accuracy and predictability improve, but tracking and auditing of deployed models becomes difficult
Solution Approach 1:
A distributed ledger system acts as an intermediary between machine-learning model updates and audit tracking. The ledger records model versions, deployment timestamps, and performance metrics immutably, enabling automatic tracking without adding operational complexity to the model update process itself.
Solution Approach 2:
Instead of tracking the actual model files directly, the system creates cryptographic hashes and metadata copies of each model version. These copies are stored in the distributed ledger, allowing efficient tracking and auditing without managing the large binary model files themselves.
2Adaptability or versatility
If multiple versions of the same machine-learning model are deployed simultaneously, then system flexibility and adaptability improve, but identification and evaluation of specific model versions becomes difficult
Solution Approach 1:
The system segments model identification into distinct components: model name, version number, deployment timestamp, and performance metrics. Each component is recorded separately in the distributed ledger, allowing precise identification and evaluation of specific versions even when multiple versions coexist.
Solution Approach 2:
The system uses cryptographic hashing to create unique digital fingerprints for each model version. These hashes act as distinctive identifiers that automatically differentiate between versions, making it easy to identify and evaluate specific models without manual tracking.
3Ease of manufacture
If manual tracking methods are used for machine-learning models, then implementation simplicity is maintained, but auditability and regulatory compliance deteriorate
Solution Approach 1:
The distributed ledger system automatically performs tracking and auditing functions without requiring manual intervention. Model deployments self-register in the ledger, and audits can be conducted by querying the ledger directly, eliminating the need for complex manual tracking processes while ensuring regulatory compliance.
Data Source
AI summary
Disclosed are various approaches for managing the status of machine-learning models using distributed ledgers. A registration request for a machine-learning model can be received. The registration request can include a model name for the machine-learning model, a version identifier for the machine-learning model, a network address from which the machine-learning model can be retrieved, a source code hash for a source code version of the machine learning model, and a runtime hash for a binary executable version of the machine-learning model. A registration identifier can then be created based at least in part on the source code hash and the runtime hash. Subsequently, an entry in the distributed ledger can be created for the machine-learning model. The entry can include the registration identifier, the model name, the model version, the network address, the source code hash, and the runtime hash.