The invention discloses a matrix decomposition-based lensless holographic microscopic speckle noise removing method and a matrix decomposition-based lensless holographic microscopic speckle noise removing device. The matrix decomposition-based lensless holographic microscopic speckle noise removing method comprises the following steps: S1, turning off a light source, and acquiring a dark field image; S2, turning on the light source, and acquiring a bright field image under uniform irradiation of the light source; S3, placing a particle-containing solution sample above a sensor, guaranteeing that the distance from the sample to the sensor is much smaller than the distance from the sample to the light source, turning on the light source, and acquiring a holographic image sequence of the sample; S4, performing flat field correction on any hologram image required to be calculated; S5, performing noise separation on the corrected holographic image by a matrix decomposition algorithm, and decomposing the corrected holographic image into two parts, namely a particle hologram part and a background noise part; S6, further performing image analysis and processing on the calculated holographic image. Through the matrix decomposition-based lensless holographic microscopic speckle noise removing method and the matrix decomposition-based lensless holographic microscopic speckle noise removing device, the speckle noise as well as interference fringe noise generated by multiple reflections of the sample can be removed, so that high-precision dynamic 3D imaging is achieved.