The invention provides a method for realizing the equalization of an FBMC system by utilizing improved sphere decoding algorithm, which comprises the following steps: by setting up an FBMC system model, and defining FBMC system parameters, analyzing the main factors of the equalization algorithm performance affecting the FBMC system; carrying out the improved sphere decoding algorithm; and carrying out simulation experiments for the algorithm and comparing; and analyzing simulation results. The improved sphere decoding algorithm means that the equalization of the FBMC system can be realized by lowering the complexity of the sphere decoding algorithm, and comprises the following steps: defining the equalization of the FBMC system; and realizing the equalization of the FBMC system by the method of probability estimate of d'2-LB (k-1) >= | | zk: M-Rk: M, K: Mbk: M | | 2 to lower the complexity of the sphere decoding algorithm. The invention utilizes the improved decoding algorithm to lower the bit error rate and the complexity of equalization algorithm of the FBMC system, and carries out verification by the simulation experiments, and the simulation results show that the improved decoding algorithm obviously lowers the complexity of the algorithm, has less loss of performance, has higher superiority and better meets the requirements of modern communications for real-time signal processing particularly when the signal to noise ratio is smaller, and has good prospects for development.