Machine Learning Model Updating Through Runtime Drift Detection

Solution Overview

Problem

Existing machine learning models degrade over time due to environmental changes, necessitating updates to maintain performance, but conventional retraining methods are costly in terms of time and resources.

Innovation Solution

Determine a performance metric and homogeneity degree of the model at runtime to identify a conceptual drift, allowing for partial or global updates based on the type of drift, thereby optimizing the update process to maintain performance efficiently.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If conventional retraining methods are used to update the machine learning model, then the model performance can be maintained, but the time cost and resource overhead are considerable

Engineering Contradiction:

Improvemodel performanceVSAvoidtime cost of model training

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The patent segments the model update process into different types (global update and partial update) based on the detected conceptual drift type. Instead of always performing a complete retraining, the system divides the update into selective partial retraining of specific model components, thereby reducing the overall time cost while maintaining performance.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent applies partial action by performing only the necessary portion of model retraining based on the detected drift type. For instance, when only output drift is detected, only the output layer is retrained rather than the entire model, thus avoiding excessive computational overhead while still maintaining model performance.

Inventive Principle:

Principle #16Partial or excessive action

2Reliability

If conventional retraining methods are used to update the machine learning model, then the model performance can be maintained, but the resource overhead is considerable

Engineering Contradiction:

Improvemodel performanceVSAvoidresource overhead of model training

Core Design Contradiction:

ReliabilityVSUse of energy by moving object

Solution Approach 1:

The patent segments the model update process into different types (global update and partial update) based on the detected conceptual drift type. Instead of always performing a complete retraining, the system divides the update into selective partial retraining of specific model components, thereby reducing the overall resource overhead while maintaining performance.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent applies partial action by performing only the necessary portion of model retraining based on the detected drift type. For instance, when only output drift is detected, only the output layer is retrained rather than the entire model, thus avoiding excessive computational overhead while still maintaining model performance.

Inventive Principle:

Principle #16Partial or excessive action

3Adaptability or versatility

If the machine learning model is updated frequently to adapt to environmental changes, then the model can maintain desirable performance, but the computational overhead increases

Engineering Contradiction:

Improvemodel adaptability to new environmentVSAvoidcomputational overhead of model update

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The patent implements a dynamic model update mechanism that adapts the update strategy based on the detected conceptual drift type. The system dynamically selects between global update and partial update approaches, and even dynamically chooses which specific components to update, thereby maintaining adaptability while optimizing computational overhead.

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The patent applies local quality by selectively updating only the specific components of the model that are affected by the detected drift, rather than uniformly updating the entire model. For example, when input drift is detected, only the input processing components are retrained, while other parts remain unchanged, thus reducing computational overhead.

Inventive Principle:

Principle #3Local quality

Data Source

PatentUS12406181B2Method, device, and computer program product for updating model

Publication Date: 2025.09.02 EMC IP HLDG CO LLC

AI summary

Embodiments of the present disclosure provide a method, a device, and a computer program product for updating a model. The method includes: determining a performance metric of a trained machine learning model at runtime; determining a homogeneity degree between a verification data set processed by the machine learning model at runtime and a training data set used to train the machine learning model; determining a type of a conceptual drift of the machine learning model based on the performance metric and the homogeneity degree; and performing an update of the machine learning model based on the type of the conceptual drift, where the update includes a partial update or a global update. In this way, a desired performance of the machine learning model can be maintained, while avoiding excessive time costs and computational resource costs caused by frequent global updates.