The invention discloses a method for improving the efficiency of erasing a floating gate, which comprises the following steps: sequentially forming a floating gate (FG) oxide layer, a FG polycrystalline silicon layer, an oxide layer-nitride layer-oxide layer (ONO) dielectric layer, a control gate (CG) polycrystalline silicon layer, a CG silicon nitride layer, a CG silicon oxide layer and a CG silicon nitride hard mask layer on a semiconductor substrate; coating photoresist on the CG silicon nitride hard mask layer, and patterning the photoresist; taking the patterned photoresist as a mask, and sequentially etching the CG silicon nitride hard mask layer, the CG silicon oxide layer, the CG silicon nitride layer, the CG polycrystalline silicon layer and the ONO dielectric layer to form two CGs; forming a CG side wall layer at two sides of each CG; forming a sacrificial layer at the outside of each CG side wall layer; taking the CG side wall layers, the sacrificial layer and the CG as masks, and etching the FG polycrystalline silicon layer to form FG; removing the sacrificial layer; and sequentially forming the oxide layer and depositing the polycrystalline silicon film outside the CG side wall layer and the FG, and finally forming the erasing gate EG by the polycrystalline silicon film. The method can effectively improve the efficiency of erasing the floating gate.