A method of forming a semiconductor device is provided. At least one stacked structure is provided on a substrate. A first spacer material layer, a second spacer material layer, and a third spacer material layer are sequentially formed on the substrate and cover the stacked structure. The first, second, and third spacer material layers are etched to form a tri-layer spacer structure on the sidewall of the stacked structure. The tri-layer spacer structure includes, from one side of the stacked structure, a first spacer, a second spacer, and a third spacer, and a dielectric constant of the second spacer is less than each of a dielectric constant of the first spacer and a dielectric constant of the third spacer.