The invention discloses a fish scale removing device and belongs to the technical field of fish product processing. The device comprises a rack, a chain conveyer, an input conveyer, a scale removing assembly and a power mechanism, wherein the chain conveyer comprises a driving chain wheel, a driven chain wheel, an annular chain and clamping mechanisms, each clamping mechanism comprises a fish clamping bottom board and a cutting board which are movably connected, short rods are arranged on the fish clamping bottom boards, the scale removing assembly comprises at least one pair of scale removing roller brushes and a shield, the input conveyer comprises a driving roller, a driven roller and an input conveying belt, inlet and outlet guide rods are arranged at an inlet and an outlet of the rack respectively, the short rods correspondingly extend out of upper parts of the inlet guide rods and extend into lower parts of the outlet guide rods, and pressing boards are also arranged on the rack. The device can remove scales continuously, realizes assembly line processing, is particularly applicable to types of fishes with flat fish bodies, and is high in scale removing efficiency and good in scale removing effect.