This invention relates to a neural video
compression method and
system based on
visual saliency, belonging to the field of
computer vision technology. Addressing the problems of existing end-to-end neural video compression methods, such as uniform bitrate allocation across regions within a frame and
neglect of differences in
human visual perception, this invention deeply integrates a lightweight
visual saliency detection network with an end-to-end video compression framework. The method includes: using a saliency detection network with MobileNetV2 as its backbone to extract multi-scale features from video frames, generating a saliency probability map, which is then concatenated with the original video frames and input into a
variable bitrate conditional
video encoding / decoding framework; adaptively allocating the encoding precision of each region based on the saliency probability map, allowing salient regions to receive more bitrate; and employing an adaptive
mask prior
entropy model for
checkerboard-style step-by-step
entropy encoding, combined with spatial and channel-dimensional adaptive quantization to improve encoding efficiency. The
system includes a saliency detection module, a
variable bitrate conditional
video encoding / decoding framework module, and an adaptive
mask prior
entropy model module, which work together to form an end-to-end optimized video
compression system. Experimental results show that, at the same bitrate, the present invention can improve human visual
image quality (EW-PSNR) by about 2 dB and save about 50% bitrate at the same quality.