Online hyperparameter tuning in distributed machine learning

Abstract

The disclosed embodiments provide a system for performing online hyperparameter tuning in distributed machine learning. During operation, the system uses input data for a first set of versions of a statistical model for a set of entities to calculate a batch of performance metrics for the first set of versions. Next, the system applies an optimization technique to the batch to produce updates to a set of hyperparameters for the statistical model. The system then uses the updates to modulate the execution of a second set of versions of the statistical model for the set of entities. When a new entity is added to the set of entities, the system updates the set of hyperparameters with a new dimension for the new entity.

Description

RELATED APPLICATION[0001]The subject matter of this application is related to the subject matter in a co-pending non-provisional application by inventors Xu Miao, Yitong Zhou, Joel D. Young, Lijun Tang and Anmol Bhasin, entitled “Version Control for Asynchronous Distributed Machine Learning,” having Ser. No. 14 / 864,474 and filing date 24 Sep. 2015 (Attorney Docket No. LI-P1583.LNK.US).BACKGROUNDField[0002]The disclosed embodiments relate to distributed machine learning. More specifically, the disclosed embodiments relate to techniques for performing online hyperparameter tuning in distributed machine learning.Related Art[0003]Analytics may be used to discover trends, patterns, relationships, and / or other attributes related to large sets of complex, interconnected, and / or multidimensional data. In turn, the discovered information may be used to gain insights and / or guide decisions and / or actions related to the data. For example, business analytics may be used to assess past performan...

AI Technical Summary

Benefits of technology

The patent text describes a system and method for performing asynchronous distributed machine learning. This involves using multiple versions of a statistical model to produce data that can be used to make informed decisions and improve performance. The system includes a server and multiple trainers that work together to collect, store, and analyze data. The technical effects of this system include improved efficiency and effectiveness in collecting, storing, and analyzing large data sets, as well as better insights and decision-making based on the data.

Problems solved by technology

However, significant increases in the size of data sets have resulted in difficulties associated with collecting, storing, managing, transferring, sharing, analyzing, and / or visualizing the data in a timely manner.

For example, conventional software tools and / or storage mechanisms may be unable to handle the petabytes or exabytes of loosely structured data that is generated on a daily and / or continuous basis from multiple, heterogeneous sources.

Images