State distribution perception sampling-based deep-value-function learning method of agent

Abstract

The invention discloses a state distribution perception sampling-based deep-value-function learning method of an agent. The method is used for more quickly learning a value function by the agent underfewer samples, and specifically includes the following steps: 1) obtaining empirical data used for learning the value function by the agent, and defining an algorithm target; 2) using a convolutionalneural network to preprocess the empirical data to obtain a more expressive feature set; 3) using an unsupervised method to cluster the empirical data set in feature space of the empirical data set;4) using a uniform sampling and cluster equal-probability sampling interpolation-based sample state distribution perception sampling method to carry out sampling according to state distribution of theempirical data set; and 5) using sampled samples by the agent for learning of the value function. The method is suitable for use in the game problem of the reinforcement learning field, and can quickly achieve a better result in a case of lesser sample quantity.

Description

Technical field [0001] The present invention belongs to the field of enhanced learning, and is a branch of the field of machine learning, and particularly relates to a sample sampling method based on empirical data state distribution perception. Background technique [0002] Sample selection is an important issue in the field of machine learning, and different selection methods directly affect the quality of model learning. In the field of reinforcement learning, sample sampling from empirical data sets can help overcome the problem of sample correlation and forgetting early samples. The goal of sample sampling is to select samples from the sample set that can accelerate the model convergence and enhance the agent's ability to perceive the environment. The traditional method generally adopts random and uniform sampling to sample from the empirical data set. This method is easy to cause sample imbalance, which makes the learning speed of the agent slow. [0003] The existing sampl...

AI Technical Summary

Problems solved by technology

However, this method does not fundamentally solve two problems: 1. The importance of samples in different states is close, but the number of generated is quite different. According to what standard to sample from the empirical data set can avoid redundant samples. Oversampling; 2. Since the sample itself is very high-dimensional, huge in number and constantly generated, it is a key factor to effectively analyze a large number of high-dimensional samples. How to efficiently sample from a large number of continuously generated sample sets

Images