The invention discloses a nested array direction-of-arrival estimation method based on off-grid sparse Bayesian learning. The method comprises: step one, matching filtering is carried out on a narrowband Gaussian signal received by a nested array to obtain a data vector x(t) including DOA information at a t time; step two, with the x(t), a received data covariance matrix R^x under a T snapshot number is calculated and vectorization is carried out on the R^x to obtain a one-dimensional data vector Y^; step three, K^ grid points theta^={theta^i}<K^>i=1 are divided uniformly in a range [-pai/2,pai/2], a counting variable I of the number of times for iteration is set to be 1, a variance vector delta and an e angle offset vector beta are initialized, and a measuring matrix phi (theta^, beta) isconstructed; step four, a variance vector theta and an angle deviation value beta in a (K^+1) dimension are updated by using an EM criterion; step five, the grid theta^ is updated by using the beta value obtained at the step four; step six, whether the counting variable I of the number of times for iteration reaches an upper limit L or the delta converges is determined; if not, the counting variable I of the number of times for iteration conforms to a formula: 1=1+1, wherein the beta is equal to 0; updating is carried out on the phi (theta^, beta) by using the updated grid theta ^, and the step four is performed; and step seven, spectral peak searching is carried out on the variance vector delta to obtain angles corresponding to K maximum points, so that a final estimation value of the target angle is obtained.