Online hyperparameter tuning in distributed machine learning

a distributed machine learning and hyperparameter tuning technology, applied in the field of distributed machine learning, can solve the problems of large data sets, difficulty in collecting, storing, sharing, analyzing, etc., and conventional software tools and/or storage mechanisms may not be able to handle petabytes or exabytes of loosely structured data generated on a daily and/or continuous basis from multiple, heterogeneous sources

Inactive Publication Date: 2018-10-04
MICROSOFT TECH LICENSING LLC
View PDF0 Cites 28 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

However, significant increases in the size of data sets have resulted in difficulties associated with collecting, storing, managing, transferring, sharing, analyzing, and / or visualizing the data in a timely manner.
For example, conventional software tools and / or storage mechanisms may be unable to handle the petabytes or exabytes of loosely structured data that is generated on a daily and / or continuous basis from multiple, heterogeneous sources.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Online hyperparameter tuning in distributed machine learning
  • Online hyperparameter tuning in distributed machine learning
  • Online hyperparameter tuning in distributed machine learning

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0012]The following description is presented to enable any person skilled in the art to make and use the embodiments, and is provided in the context of a particular application and its requirements. Various modifications to the disclosed embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments and applications without departing from the spirit and scope of the present disclosure. Thus, the present invention is not limited to the embodiments shown, but is to be accorded the widest scope consistent with the principles and features disclosed herein.

[0013]The data structures and code described in this detailed description are typically stored on a computer-readable storage medium, which may be any device or medium that can store code and / or data for use by a computer system. The computer-readable storage medium includes, but is not limited to, volatile memory, non-volatile memory, magnetic and optical...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The disclosed embodiments provide a system for performing online hyperparameter tuning in distributed machine learning. During operation, the system uses input data for a first set of versions of a statistical model for a set of entities to calculate a batch of performance metrics for the first set of versions. Next, the system applies an optimization technique to the batch to produce updates to a set of hyperparameters for the statistical model. The system then uses the updates to modulate the execution of a second set of versions of the statistical model for the set of entities. When a new entity is added to the set of entities, the system updates the set of hyperparameters with a new dimension for the new entity.

Description

RELATED APPLICATION[0001]The subject matter of this application is related to the subject matter in a co-pending non-provisional application by inventors Xu Miao, Yitong Zhou, Joel D. Young, Lijun Tang and Anmol Bhasin, entitled “Version Control for Asynchronous Distributed Machine Learning,” having Ser. No. 14 / 864,474 and filing date 24 Sep. 2015 (Attorney Docket No. LI-P1583.LNK.US).BACKGROUNDField[0002]The disclosed embodiments relate to distributed machine learning. More specifically, the disclosed embodiments relate to techniques for performing online hyperparameter tuning in distributed machine learning.Related Art[0003]Analytics may be used to discover trends, patterns, relationships, and / or other attributes related to large sets of complex, interconnected, and / or multidimensional data. In turn, the discovered information may be used to gain insights and / or guide decisions and / or actions related to the data. For example, business analytics may be used to assess past performan...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(United States)
IPC IPC(8): G06N7/02G06N3/04G06F15/18G06N3/08G06N99/00G06F17/30G06N20/00
CPCG06N7/023G06N7/026G06N3/0436G06F17/30554G06N3/08G06N99/005G06F15/18G06F15/76G06N3/006G06N3/082G06N20/10G06N20/20G06N20/00G06N5/01G06N7/01G06F16/248
Inventor WOOD, IAN B.MIAO, XUTSAI, CHANG-MINGYOUNG, JOEL D.
Owner MICROSOFT TECH LICENSING LLC
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap