Variance reduction optimization method based on large-scale machine learning

A technology of machine learning and optimization methods, applied in the field of artificial intelligence, can solve problems such as difficult loss function gradients

Pending Publication Date: 2020-10-27
XIDIAN UNIV
View PDF0 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

In addition, Point-SVRG also made great changes to the algorithm framework of the variance reduction stochastic gradient descent algorithm SVRG. During the inner loop iteration process, Point-S

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Variance reduction optimization method based on large-scale machine learning
  • Variance reduction optimization method based on large-scale machine learning

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0019] Below with reference to accompanying drawing, the present invention is described in detail:

[0020] refer to figure 1 , the present invention provides a variance reduction optimization algorithm based on large-scale machine learning, the specific implementation steps are as follows:

[0021] Step 1: Select the iteration number m of the inner loop, the iteration number k of the outer loop, and the iterative learning rate γ, and the iterative learning rate γ is less than μ / L(L-μ), where L represents the smooth condition of the sample function, and μ represents The strong convex condition of the sample function; the number of iterations m of the inner loop should be selected as close to the total number of sample functions as possible, but not exceed the total number of samples n;

[0022] Step 2: Select an initial point. The initial point is generally given according to historical experience or can be obtained according to simple calculations, as close to the optimal va...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a variance reduction optimization method based on large-scale machine learning, which mainly solves the problems that in the prior art, the variance is too large in the randomoptimization iteration process, so that the optimal value is not converged, and the convergence speed is not high enough, and the invention adopts the technical scheme that the method comprises the following steps: 1) setting a starting point and determining the number of iterations; 2) assigning an initial point to an internal loop as a starting point, and performing full-gradient calculation; 3)randomly selecting samples from the large samples to perform variable control operation; according to the invention, it is ensured that the function achieves the linear convergence effect of the acceleration property under the smooth and strong convex conditions, the optimal value is rapidly and accurately solved, gradient is not required in the main inner loop iteration process, and the method is suitable for application scenes of large-scale machine learning.

Description

technical field [0001] The invention belongs to the field of artificial intelligence, and further relates to machine learning technology, specifically a variance reduction optimization method based on large-scale machine learning, which can be used to quickly and accurately solve optimal values ​​in large-scale machine learning application scenarios. Background technique [0002] In recent years, the vigorous development of artificial intelligence has attracted more and more attention from academia and industry. Since the 1950s, people have been hoping that the classic artificial intelligence technology based on logic, knowledge representation, reasoning and planning can bring about revolutionary changes, and the emerging artificial intelligence technology can understand human language, control robots, and serve people. Daily life and industrial production provide decision-making and advice. Many researchers work on the design of systems based on statistical techniques, for...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
IPC IPC(8): G06N20/00
CPCG06N20/00
Inventor 王子龙丘炜
Owner XIDIAN UNIV
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap