A ballastless track construction method with sleepers arranged firstly and rail panel assembled later comprises the steps of construction layout, installation of steel bars at a bottom layer of a track bed, dispersion of the sleepers and spacing adjustment, assembly of hoisted rail panel frames, coarse adjustment of rail panels, installation of steel bars at top layers of track bed boards, installation of longitudinal and transverse formworks, comprehensive grounding welding, insulating property test, fine adjustment of the rail panels, concrete pouring construction, concrete curing and removal of bent frame molds. According to the construction method, a method that a gantry crane is used for assembling the rail panels on a mobile assembly platform is changed, a method that the sleepers are arranged firstly and the rail panels are assembled later on a construction operation surface is adopted directly, the usage of the mobile assembly platform is canceled, the problem of land occupation of the mobile assembly platform is solved, and the field change efficiency of other machines, tools and equipment is improved, in addition, a crane instead of the gantry crane is used for matching assembly, so that the gantry crane is mainly used for transferring other light machines, tools and equipment, the safety of transferring hoisting is improved, the continuous construction is guaranteed, the effective utilization rate of the gantry crane is increased, and the construction efficiency is improved.