The invention discloses an optical transmission method and device based on power spectrum shaping. Direct detection, namely square demodulation, of high-spectrum-efficiency optical signals is supported. According to the invention, a power spectrum shaping modulation method is adopted at a sending end, and the modulation method realizes multi-level improved dual-binary coding and multi-level offsetdual-binary coding by biasing a Mach-Zehnder optical modulator at a zero point, namely Vpi, and increasing level in combination with nonlinear precoding, and can also realize residual sideband improved dual-binary coding in combination with an optical filter; inter-symbol interference of received signals is removed at a receiving end by adopting nonlinear equalization, and meanwhile, the difference of received levels can be fed back to a transmitter, so that non-equidistant multi-level modulation is realized, and the nonlinear equalization complexity is reduced. According to the method, signals needing coherent demodulation can be directly detected through proper nonlinear precoding and nonlinear equalization, so that the hardware complexity is reduced.