Underwater image compression method and device under extremely low bit rate and storage medium

Abstract

The invention relates to an underwater image compression method and device under an extremely low bit rate and a storage medium. According to the method, after an original underwater image is acquired, a foreground image and a semantic graph are extracted from the original underwater image, a decoded foreground image is acquired by using a preset foreground image high-pixel fidelity compressor based on the foreground image, and the decoded foreground image is acquired based on the semantic graph and the decoded foreground image. Obtaining a reconstructed underwater image by using a preset background image high-perception fidelity compressor; wherein the foreground image high-pixel fidelity compressor comprises a foreground target encoder and a foreground target decoder which are connected in sequence, the foreground target decoder comprises an underwater prior guidance module, and the background image high-perception fidelity compressor comprises a semantic graph encoder-decoder and a foreground-assisted background generation network which are connected in sequence. Compared with the prior art, the method has the advantages that the high-perception fidelity quality of the whole underwater image and the high-pixel fidelity quality of the foreground target are ensured, and the compression efficiency of the underwater image is improved.

Description

Technical Field

[0001] The present invention relates to the field of image coding technology, and in particular to an underwater image compression method, device and storage medium at an extremely low bit rate. Background Art

[0002] In underwater wireless acoustic channels with extremely limited bandwidth, underwater images must be compressed to very low bit rates. For example, in a common underwater acoustic channel with a bandwidth of 15kbps, to transmit a 512×384 pixel underwater image within 3 seconds, the compression bit rate (i.e., the number of bits per pixel, bpp) needs to be about 0.2. However, at such a low bit rate, the existing image compression algorithms will significantly reduce the pixel fidelity and visual perception quality of the decoded image. Standard codecs such as JPEG, JPEG2000, HEVC-based BPG and VVC-intra are mainly designed for conventional terrestrial scenes, and it is difficult to achieve satisfactory reconstruction quality under extremely low bit rate conditions.

[0003] In recent years, in order to achieve visually pleasing reconstruction effects at extremely low bit rates, a variety of end-to-end extreme image compression methods based on Generative Adversarial Networks (GAN) have been proposed. Rippel et al. first introduced adversarial loss in deep compression systems to minimize the distribution difference between the decoded image and the original image, thereby improving the visual quality. Agustsson et al. applied advanced GAN technology to the popular compressed autoencoder (AE), achieving perceptual quality significantly better than BPG at extremely low bit rates. Mentzer et al. enhanced the perceptual quality of AE reconstructed images through conditional GAN ​​(cGAN) and introduced the perceptual distortion index LPIPS as a loss function, achieving the state-of-the-art (SOTA) extreme compression performance. Although these GAN-based methods perform well in terms of perceptual quality, they still have shortcomings in the pixel fidelity of the reconstructed images. In addition, these methods usually do not take into account the unique non-Gaussian data distribution characteristics of underwater images, but use Gaussian distribution models to predict the distribution probability of latent features for entropy coding, resulting in low compression efficiency of underwater images. Summary of the invention

[0004] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and to provide an underwater image compression method, device and storage medium at an extremely low bit rate. The method constructs an efficient underwater image compression framework at an extremely low bit rate, which can ensure the high perceptual fidelity quality of the entire underwater image and the high pixel fidelity quality of the foreground target.

[0005] The purpose of the present invention can be achieved by the following technical solutions:

[0006] According to a first aspect of the present invention, there is provided an underwater image compression method at an extremely low bit rate, comprising the following steps: an original image acquisition step, acquiring an original underwater image; a foreground compression step, extracting a foreground image from the original underwater image, and based on the foreground image, obtaining a decoded foreground image using a preset foreground image high-pixel-fidelity compressor; a background compression and underwater image reconstruction step, extracting a semantic map from the original underwater image, and based on the semantic map and the decoded foreground image, obtaining a reconstructed underwater image using a preset background image high-perceptual-fidelity compressor; wherein the foreground image high-pixel-fidelity compressor comprises a foreground target encoder and a foreground target decoder connected in sequence, the foreground target decoder comprises an underwater priori guidance module, and the background image high-perceptual-fidelity compressor comprises a semantic map encoder-decoder and a foreground-assisted background generation network connected in sequence.

[0007] As a preferred technical solution, the foreground compression step specifically includes: based on the original underwater image, using a preset underwater prior extraction network to obtain underwater physical prior information; using the foreground target mask obtained based on the semantic graph to preprocess the original underwater image to obtain a foreground image; based on the foreground image, using the foreground target encoder to extract potential features, and generating a bit stream through quantization and arithmetic coding; based on the generated bit stream and the underwater physical prior information, using the foreground target decoder to obtain a decoded foreground image.

[0008] As a preferred technical solution, the pretreatment process is expressed as:

[0009] I FGOs =I whole ×M

[0010] In the formula, I FGOs represents the foreground image, I whole represents the entire original underwater image, and M represents the foreground target mask.

[0011] As a preferred technical solution, the quantization process is implemented using a uniform noise approximation method.

[0012] As a preferred technical solution, the background compression and underwater image reconstruction steps specifically include: based on the original underwater image, using a preset semantic segmentation network to obtain a corresponding semantic map; assigning values ​​to the foreground area and the background area in the semantic map respectively to obtain a foreground target mask; based on the semantic map, using the semantic map encoder-decoder to perform encoding and decoding in sequence; based on the decoded semantic map and the decoded foreground image, using the foreground-assisted background generation network to obtain a reconstructed underwater image.

[0013] As a preferred technical solution, the process of obtaining the foreground object mask specifically includes: assigning a value of 1 to the foreground area in the semantic map, and assigning a value of 0 to the background area in the semantic map.

[0014] As a preferred technical solution, the foreground-assisted background generation network includes a generator and a discriminator connected in sequence, the generator is used to splice the decoded semantic map with the decoded foreground image, and the discriminator is used to supervise and assist the generator in optimizing the quality of the reconstructed underwater image.

[0015] As a preferred technical solution, the hybrid loss function used by the foreground-assisted background generation network is expressed as:

[0016]

[0017] In the formula, k ′ =10, and

[0018]

[0019] In the formula, ||·||1 represents the L1 norm, represents the GAN loss function used to optimize the generator G and the discriminator D, represents the feature matching loss function, is the kth discriminator D k The i-th layer feature extractor, n is the total number of layers, N i is the number of elements in each layer, x is the original underwater image input, c is the preset discriminator input condition, represents the perceptual loss function, V (j) is the j-th layer feature tensor of the pre-trained VGG-19 network, x' represents the reconstructed image, m represents the number of network layers, represents the pixel-wise loss function.

[0020] According to a second aspect of the present invention, there is provided an underwater image compression device at an extremely low bit rate, comprising a memory, a processor, and a program stored in the memory, wherein the processor implements the method when executing the program.

[0021] According to a third aspect of the present invention, there is provided a storage medium having a program stored thereon, wherein the program implements the method described above when executed.

[0022] Compared with the prior art, the present invention has the following beneficial effects:

[0023] 1. The underwater image compression method proposed in the present invention is suitable for underwater image reconstruction at extremely low bit rates. A compression framework is established to achieve the best high-perceptual fidelity quality of the entire image and high-pixel fidelity quality of the foreground image at extremely low bit rates. Based on the framework, a foreground image high-pixel fidelity compressor with an underwater prior guidance module is used to perform high-pixel fidelity compression on the foreground image, and a foreground-assisted background generation network is used to synthesize a realistic background from a semantic map. The underwater prior can be used to adjust the Gaussian feature distribution of the receiving end to make the image more consistent with the underwater image distribution, effectively improve the pixel fidelity of the foreground image, and effectively save the number of bits, avoid the problem of single color of the synthesized background, and use the foreground target with high-fidelity information as auxiliary information to guide the background synthesis, thereby significantly improving the perceptual quality of the synthesized background.

[0024] 2. The present invention takes into account the different tasks of foreground images and background images in practical applications, adopts a differential compression method for foreground and background, processes the foreground and background separately, thereby effectively saving a large amount of bit stream;

[0025] 3. The present invention introduces an underwater prior guidance module in the foreground object decoder, combines the underwater physical prior information, and uses the module to adjust the Gaussian distribution of the foreground image features to make them more consistent with the real underwater image, thereby improving the reconstruction quality of the foreground image, and implicitly reduces the coding feature bits through end-to-end training, thereby effectively improving the compression efficiency of the underwater image;

[0026] 4. The present invention splices the decoded semantic map and the decoded foreground image together as paired inputs of the foreground-assisted background generation network. Under the constraint of perceptual consistency between the foreground target and the generated background, it can effectively improve the perceptual quality of the background generated by the semantic map, making it more natural and realistic. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 A schematic diagram of the overall network structure of the method provided by the present invention;

[0028] Figure 2 Schematic diagram of the structure of a high pixel fidelity compressor for foreground images in an embodiment of the present invention;

[0029] Figure 3 A schematic diagram of the structure of a foreground-assisted background generation network in an embodiment of the present invention;

[0030] Figure 4Schematic diagram of the comparison of distribution histograms of general atmospheric natural images and underwater images in an embodiment of the present invention;

[0031] Figure 5 The figure is a visual comparison of underwater images reconstructed by different algorithms in the embodiments of the present invention. DETAILED DESCRIPTION

[0032] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0033] Example

[0034] The following considerations are made with respect to the defects of the aforementioned prior art in processing underwater images at extremely low bit rates: Compared with images in the natural atmosphere, underwater images usually exhibit a bluish or greenish hue and a lower contrast. This is due to wavelength-dependent attenuation of light when it propagates in water. According to common underwater physical imaging models, the underwater imaging process is affected by physical priors such as the attenuation coefficient of water quality and ambient light. Therefore, by introducing these underwater physical priors into the end-to-end compression model, the Gaussian distribution of latent features can be adjusted to make it more consistent with real underwater images, thereby improving the compression efficiency of underwater images.

[0035] Based on this, the present embodiment provides an underwater image compression method at an extremely low bit rate, including: an original image acquisition step, acquiring an original underwater image; a foreground compression step, extracting a foreground image from the original underwater image, and based on the foreground image, obtaining a decoded foreground image using a preset foreground image high-pixel fidelity compressor; and a background compression and underwater image reconstruction step, extracting a semantic map from the original underwater image, and based on the semantic map and the decoded foreground image, obtaining a reconstructed underwater image using a preset background image high-perceptual fidelity compressor; wherein the foreground image high-pixel fidelity compressor includes a foreground target encoder and a foreground target decoder connected in sequence, the foreground target decoder includes an underwater prior guidance module, and the background image high-perceptual fidelity compressor includes a semantic map encoder-decoder and a foreground-assisted background generation network connected in sequence. The overall network structure based on this method is as follows: Figure 1As shown in the figure, UPGAE-FGO Encoder (Underwater Prior Guided AutoEncoder-ForeGround Object Encoder) is the underwater prior-guided foreground target encoder, UPGAE-FGO Decoder is the underwater prior-guided foreground target decoder, BGGAN (BackGround Generative Adversarial Network) is the foreground-assisted background generation network, Q represents quantization, AE represents arithmetic coding, and AD represents arithmetic decoding.

[0036] like Figure 2 As shown, the foreground image high pixel fidelity compressor includes a preprocessing module, a foreground target encoder and a foreground target decoder connected in sequence, and the foreground target decoder includes an underwater prior guidance module (i.e., UPG Block). The background image high perceptual fidelity compressor includes a semantic graph encoder-decoder and a foreground-assisted background generation network connected in sequence. Figure 3 As shown, the background generation network includes a generator G and a discriminator D. The generator G is used to splice the decoded semantic map with the decoded foreground image, and the discriminator D is used to supervise and assist the generator G to optimize the quality of the reconstructed underwater image.

[0037] Based on the foreground image high pixel fidelity compressor and the background image high perceptual fidelity compressor, the specific execution process of the foreground compression step, the background compression step and the underwater image reconstruction step is as follows:

[0038] First, the original underwater image is input into the preset underwater prior extraction network to extract underwater physical prior information (such as attenuation coefficient and ambient light). At the same time, the same original underwater image is input into the preset underwater semantic segmentation network to obtain a semantic map. The foreground target mask is generated by assigning the foreground area in the semantic map to "1" and the background area to "0". When compressing the foreground image, the foreground image is obtained by preprocessing the entire image using the foreground mask M. The preprocessing process is as follows:

[0039] I FGOs =I whole ×M (1)

[0040] In the formula, I FGOs represents the foreground image, I whole represents the entire original underwater image, and M represents the foreground target mask.

[0041] Next, the background is filled with the average pixel value of the foreground area to form a smooth transition between the foreground and background, reducing the consumption of coding bits. After that, the processed foreground image is input into the foreground encoder to extract the latent features and generate the bitstream through quantization and arithmetic coding. The quantization adopts the uniform noise approximation method: uniform noise is added in the training phase, while each element is quantized to the nearest integer using rounding in the test phase. At the decoding end, after the generated bitstream is arithmetically decoded, the foreground features of the Gaussian distribution are adjusted by the foreground target decoder based on the underwater physical prior information to make it more consistent with the distribution of the real underwater image, thereby improving the pixel fidelity of the foreground image.

[0042] In the background compression part, after obtaining the semantic map using the preset underwater semantic segmentation network, the semantic map is first encoded and decoded in sequence using the semantic map encoder-decoder, and then the decoded semantic map is sent to the foreground-assisted background generation network (BGGAN) to generate realistic background content. By optimizing the background synthesis using the foreground information with high pixel fidelity, the perceptual quality of the synthesized background is improved.

[0043] Next, the foreground image high pixel fidelity compressor and the background image high perceptual fidelity compressor are introduced in detail respectively.

[0044] like Figure 2 As shown in the figure, in the process of foreground compression, the autoencoder is selected as the foreground compressor because the autoencoder based on deep learning can achieve more ideal results in image compression than the traditional codec. In the figure, Q, AE and AD represent quantization, arithmetic coding and arithmetic decoding respectively, σ represents the variance of the Gaussian model, GDN represents generative split normalization, IGDN is the inverse GDN, Conv represents convolution, UPG Block represents the underwater prior guidance module, Abs represents the absolute value function, ReLU represents the activation function, Exp represents the expectation function, x, x f , are the input original underwater image, the preprocessed foreground image, and the reconstructed foreground image, respectively. f , z, z f , Represents the corresponding output image after different operations. Different from conventional autoencoders, which are usually used for compressing atmospheric natural images, the pixel fidelity compressor for the foreground proposed in this embodiment introduces underwater physical prior information through the UPG module guided by underwater prior to adjust the distribution of Gaussian latent features to make them more consistent with the characteristics of underwater images.

[0045] Generally speaking, the autoencoder is optimized through the rate-distortion loss, which can be described as:

[0046]

[0047] Where λ is a parameter that balances bitrate and distortion, D is a distortion metric such as mean square error (MSE), and R represents the bitrate approximated by the entropy of the latent features. The true probability distribution q of the latent features for entropy coding is unknown. A common solution is to use a mathematical model to estimate the probability distribution. Usually, the Laplace model or the more efficient Gaussian model is used, and then the bitrate R is calculated using the cross entropy loss:

[0048]

[0049] In the formula, the first term is Shannon entropy, and the second term is Kullback-Leibler divergence (KL), which is used to measure the relationship between q and In order to reduce R, KL can be regarded as a regularizer to make the true distribution q close to the estimated distribution based on the Gaussian model. Therefore, the latent features after network optimization clearly conform to the Gaussian distribution.

[0050] Although Gaussian distribution has been shown to be effective in modeling general atmospheric natural images, it is not ideal when dealing with underwater images. Figure 4 As shown, Figure 4 (a) is the distribution of natural images, and (b) is the distribution of underwater images. By studying the distribution of 1500 atmospheric natural images randomly selected from the ImageNet dataset and 1500 underwater images randomly selected from the underwater dataset, it can be observed that the distribution of atmospheric natural images generally follows a generalized quasi-Gaussian distribution, while the distribution of underwater images is different. Therefore, using Gaussianized features to reconstruct non-Gaussian distributed underwater images is an indirect and challenging problem for the decoder, which often leads to poor reconstruction quality. In addition, poor reconstruction quality forces the autoencoder (AE) to encode more features and consume more bits to reconstruct better quality images, resulting in low compression efficiency of underwater images.

[0051] In order to solve this problem, according to the physical model of underwater imaging, it is found that non-Gaussian distributed underwater images are evolved from general Gaussian images through the underwater imaging process. Inspired by this discovery, physical priors are used to simulate the underwater imaging process, and the received Gaussian potential features are adjusted to make them more consistent with the distribution of underwater images. The physical model of underwater imaging can be defined as:

[0052] I=J·e -βd +α·(1-e -βd ) (4)

[0053] Where I represents the captured underwater image, J is the real scene brightness, and d represents the distance from the camera to the object. α represents the ambient light, β refers to the attenuation coefficient of the water medium, and since the depth between the foreground and the camera is basically the same, the influence of d can be ignored and d is set to a constant. In this underwater imaging physical model, the image of the real scene does not have the characteristics of underwater images, such as color shift and low contrast, but follows a generalized Gaussian-like distribution similar to atmospheric natural images. However, due to the influence of physical priors in the underwater imaging process (such as ambient light and attenuation coefficient), the distribution of the captured underwater image is different from that of the atmospheric natural image. Therefore, under the influence of underwater physical priors, the distribution of underwater images can be regarded as evolving from the Gaussian distribution of atmospheric natural images.

[0054] Based on this, an underwater prior-guided structure is designed in the high-pixel fidelity compressor for foreground images. This structure introduces underwater physical prior information such as ambient light and attenuation coefficient to correct the distribution of Gaussian latent features to make it more consistent with the real underwater image distribution. Through the end-to-end rate-distortion optimization strategy of the autoencoder, this structure not only improves the quality of the reconstructed image, but also intelligently reduces the use of coding bits, thereby effectively improving the compression efficiency of underwater images.

[0055] In addition to the foreground area that needs high-fidelity compression, the relatively unimportant background also needs to be compressed to ensure that it meets the human visual experience. Given that the semantic map can be compressed to a very small bitstream, the method of this embodiment uses the semantic map as the input of the generative neural network and synthesizes a background image with high perceptual similarity under the guidance of the high-pixel-fidelity foreground image.

[0056] The whole background compression process is as follows Figure 1 As shown in the figure, it mainly consists of two parts: semantic graph encoder-decoder and foreground auxiliary background generative network. First, the semantic graph is compressed using the screen content codec (SCC). At the decoding end, the decoded semantic graph is input into the background generative network as auxiliary information to guide the generation of the background.

[0057] Figure 3 The structure of the background generative network is shown, where "Conv" and "DeConv" represent convolution and deconvolution operations respectively, "SPADE" represents an effective normalization layer (Spatially-Adaptive Denormalization), "IN" refers to instance normalization, "Num" represents the number of residual modules in the network, Stride represents the step size, and k represents the kernel size. Through this structure, the background generative network can synthesize a background with higher perceptual quality under the guidance of the foreground, further improving the visual effect of the overall image.

[0058] The background generative network consists of a generator G and a discriminator D. The structure of the generator G includes a global generator G1 and a local generator G2. Among them, SPADE (Spatially-Adaptive Denormalization) is an efficient normalization layer that exhibits excellent pixel-by-pixel guidance capabilities in semantic image synthesis.

[0059] The discriminator D is a multi-scale discriminator that downsamples the reconstructed image and the original image by 2 times to form an image pyramid of two scales, and then sends these images to two discriminators of the same structure for discrimination. Through this multi-scale design, the discriminator can use two different receptive fields to more accurately predict the authenticity of a given image, thereby improving the ability to judge the details and overall structure of the image. In this way, the generator G can generate high-quality background images under the supervision of the discriminator D, which can be better integrated with the foreground image and improve the overall perceptual quality of the image.

[0060] Different from traditional generative networks that directly take noise vectors or semantic maps as input, background generative networks concatenate the decoded foreground image with the semantic map as paired input. This approach improves the perceptual quality of the generated images in two ways.

[0061] First, this method can make the generated background color more accurate. Traditional generative networks cannot obtain detailed color information from semantic maps, and usually generate a single and repeated color for each semantic. For example, even if the hue of seawater is diverse in reality, the seawater generated by traditional generative networks may always be blue because most seawater images in the training set are blue. By inputting foreground information into the generator, the generator can automatically extract color information from the foreground under the constraint of L1 norm loss, thereby generating a background with the correct hue.

[0062] Secondly, this method makes the transition between background and foreground more natural. Under the optimization of global perceptual loss, the generated background texture can match the foreground texture well, producing a smooth transition. Therefore, the spliced ​​input of the foreground image and the semantic map significantly improves the perceptual quality of the generated image, making the transition between foreground and background more natural.

[0063] To optimize the background synthesis process, the following loss function is used:

[0064] (1) It is to optimize the GAN loss function of the generator and the discriminator, expressed as:

[0065]

[0066] In the formula, The calculation formula is:

[0067]

[0068] Wherein, c is the preset discriminator input condition, and in this embodiment, the input condition is the original image.

[0069] (2) is the feature matching loss function, which extracts features from multiple layers of the discriminator and forces these intermediate representations to match in real and synthetic images, thereby stabilizing training, expressed as:

[0070]

[0071] In the formula, is the kth discriminator D k The i-th layer feature extractor, n is the total number of layers, N i is the number of elements in each layer.

[0072] (3) is the perceptual loss function, which is used to smooth the background and foreground and improve the global perceptual consistency, expressed as:

[0073]

[0074] Where V (j) (·) is the j-th layer feature tensor of the pre-trained VGG-19 network, x' is the reconstructed image, and m is the number of network layers.

[0075] (4) is used to measure the pixel-by-pixel loss function, which can significantly improve the perceptual similarity between the reconstructed image and the label image, expressed as:

[0076]

[0077] In general, the BGGAN designed in this embodiment aims to minimize the hybrid loss function Defined as:

[0078]

[0079] In the formula, k ′ =10.

[0080] Next, the effectiveness of the method provided in this embodiment is demonstrated through experiments:

[0081] 1) Dataset

[0082] This embodiment conducts experiments on four real underwater image datasets, including the SUIM dataset, EUVP dataset, UFO dataset, and UIEBD dataset. The SUIM dataset contains various underwater scenes collected in ocean exploration and human-machine collaboration experiments, covering eight object categories such as fish, ruins, AUV (autonomous underwater vehicle), human divers, water bodies, rocks, coral reefs, and seaweed, each with pixel-level semantic annotations. The SUIM dataset includes 1,600 underwater images and can be flexibly divided into foreground targets and backgrounds according to practical applications. In this experiment, fish, ruins, AUVs, and human divers are classified as foreground targets, and the remaining categories belong to the background. In order to train the network, the SUIM dataset is divided into 1,459 training pairs and 100 test pairs, with an image resolution of 512×384. The training dataset is data augmented by random flipping and cropping before each training epoch. In addition, in order to verify the generalization ability of the network, the trained network is tested on three other underwater datasets (EUVP, UFO, and UIEBD).

[0083] 2) Comparison of algorithms

[0084] On the basis of the above, the method provided in this embodiment is compared with three other existing methods, including the traditional standard codec, the common underwater image compression method and the optimal extreme compression method. Specifically:

[0085] Standard codecs: including the latest VVC-intra, popular BPG (HEVC-intra), and JPEG2000. For VVC-intra and BPG (HEVC-intra), the highest image setting of 4:4:4 was used. In addition, the performance of JPEG2000 in ROI mode was also compared.

[0086] Underwater image compression algorithms: including the Zhuang algorithm driven by MS-SSIM and the He algorithm driven by PSNR.

[0087] Extreme compression method: including the GAN-based extreme compression method proposed by Mentzer et al.

[0088] 3) Evaluation indicators

[0089] All methods are evaluated by calculating the compression bitrate (bpp) and quality loss. Quality loss indicators include PSNR, Perceptual Index (PI), Natural Image Quality Evaluator (NIQE), and Fréchet Inception Distance (FID) to compare image quality. In addition, human eye scoring evaluation is also performed to qualitatively verify that the proposed algorithm achieves better visual quality than other methods. Among these distortion indicators, PSNR is used to evaluate the pixel fidelity of the foreground object, and PI, NIQE, and FID are used to evaluate the perceptual quality of the entire image.

[0090] 4) Comparison of perceived quality of the entire image

[0091] (4.1) Quantitative comparison: The perceptual quality comparison results on the SUIM test set are shown in Table 1, where the lower the PI / NIQE / FID, the better the perceptual quality. It can be seen that the method in this embodiment achieves better perceptual quality in terms of PI / NIQE / FID, and the pixel fidelity of the foreground object (PSNRFGO) is higher than all the compared methods.

[0092] Table 1 Quantitative quality comparison of the entire image in the SUIM test set

[0093]

[0094] (4.2) Qualitative comparison: Figure 5 The following figure shows the image examples compressed by the method of this embodiment, VVC-intra, Mentzer and Zhuang algorithms. It can be seen from the figure that the underwater image reconstructed by the method of this embodiment has more realistic quality and clearer texture. Specifically:

[0095] Zhuang algorithm: Since the compression ratio of the color channel is 4 times higher than that of the brightness channel, this results in serious loss of color information in certain locations at extremely low bit rates, causing obvious visual distortion.

[0096] Mentzer's GAN-based method: Although this method can reconstruct more texture details, the texture and details are not realistic, such as the texture of the fish in the 4th row of the figure.

[0097] VVC-intra: At low bit rates, the images reconstructed by VVC-intra are blurred and have block effects, which seriously affects the visual perception quality.

[0098] The method of this embodiment: In comparison, the reconstructed image of PPECN has more realistic and clearer texture, showing higher visual quality.

[0099] (4.3) Subjective quality evaluation: According to the human eye scoring process commonly used in perceptual limit compression, 20 users were randomly selected to score their preferences for each image group. The questionnaire included 90 groups of images, each with 6 images, which were reconstructed images of the algorithm of the present invention and the five comparison methods at a bpp of 0.045. Users were asked to evaluate which method had better visual quality in terms of reconstruction results on a scale of 1 to 6.

[0100] The results show:

[0101] PPECN had the highest average ranking of 1.78.

[0102] Mentzer's average ranking is 2.07.

[0103] BPG's average ranking is 3.25.

[0104] The average rank of JPEG2000 is 3.86.

[0105] Zhuang's average ranking is 4.73.

[0106] He has an average rank of 5.29.

[0107] These results indicate that more users believe that the reconstructed images of the present invention have better perceptual quality.

[0108] In summary, a large number of standard test set experimental results show that the present invention exhibits better perceptual quality in all indicators. Quantitative comparison shows that the pixel fidelity of the foreground target is better than all the comparison methods, while qualitative comparison shows more realistic and clearer textures. In the subjective quality evaluation, more users believe that the reconstructed image of the present invention has higher perceptual quality, which verifies that the present invention can reconstruct underwater images with higher perceptual similarity at extremely low bit rates.

[0109] Furthermore, this embodiment also provides an underwater image compression device at an extremely low bit rate, including a memory, a processor, and a program stored in the memory. When the processor executes the program, one or more steps of the aforementioned method are implemented. The specific execution steps are basically the same as the implementation process of each step of the aforementioned method, and will not be repeated here.

[0110] Furthermore, this embodiment also provides a storage medium on which a program is stored. When the program is executed, one or more steps of the aforementioned method are implemented. The specific execution steps are basically the same as the implementation process of each step of the aforementioned method, and will not be repeated here.

[0111] The preferred specific embodiments of the present invention are described in detail above. It should be understood that a person skilled in the art can make many modifications and changes based on the concept of the present invention without creative work. Therefore, any technical solution that can be obtained by a person skilled in the art through logical analysis, reasoning or limited experiments based on the concept of the present invention on the basis of the prior art should be within the scope of protection determined by the claims.

Claims

1. A method for compressing underwater images at an extremely low bit rate, characterized in that: The following steps are involved: An original image acquisition step, acquiring an original underwater image; a foreground compression step, extracting a foreground image from the original underwater image, and based on the foreground image, obtaining a decoded foreground image using a preset foreground image high pixel fidelity compressor; Background compression and underwater image reconstruction step, extracting a semantic map from the original underwater image, and obtaining a reconstructed underwater image using a preset background image high perception fidelity compressor based on the semantic map and the decoded foreground image; Among them, the foreground image high pixel fidelity compressor includes a foreground target encoder and a foreground target decoder connected in sequence, the foreground target decoder includes an underwater prior guidance module, and the background image high perceptual fidelity compressor includes a semantic graph encoder-decoder and a foreground-assisted background generation network connected in sequence.

2. The underwater image compression method at an extremely low bit rate according to claim 1, characterized in that: The foreground compression step specifically includes: Based on the original underwater image, using a preset underwater prior extraction network to obtain underwater physical prior information; Preprocessing the original underwater image using the foreground target mask obtained based on the semantic map to obtain a foreground image; Based on the foreground image, extracting potential features using the foreground target encoder, and generating a bit stream through quantization and arithmetic coding; Based on the generated bit stream and the underwater physical prior information, the foreground object decoder is used to obtain a decoded foreground image.

3. The underwater image compression method at an extremely low bit rate according to claim 2, characterized in that: The pre-processing process is expressed as: I FGOs =I whole ×M In the formula, I FGOs represents the foreground image, I whole represents the entire original underwater image, and M represents the foreground target mask.

4. The underwater image compression method at an extremely low bit rate according to claim 2, characterized in that: The quantization process is implemented using the uniform noise approximation method.

5. The underwater image compression method at an extremely low bit rate according to claim 1, characterized in that: The background compression and underwater image reconstruction steps specifically include: Based on the original underwater image, a corresponding semantic map is obtained using a preset semantic segmentation network; Assigning values ​​to the foreground area and the background area in the semantic map respectively to obtain a foreground object mask; Based on the semantic graph, sequentially encoding and decoding are performed using the semantic graph encoder-decoder; Based on the decoded semantic graph and the decoded foreground image, the foreground-assisted background generation network is used to obtain a reconstructed underwater image.

6. The underwater image compression method at an extremely low bit rate according to claim 5, characterized in that: The process of acquiring the foreground object mask specifically includes: assigning a value of 1 to the foreground area in the semantic map, and assigning a value of 0 to the background area in the semantic map.

7. The underwater image compression method at an extremely low bit rate according to claim 5, characterized in that: The foreground-assisted background generation network includes a generator and a discriminator connected in sequence, the generator is used to splice the decoded semantic map with the decoded foreground image, and the discriminator is used to supervise and assist the generator in optimizing the quality of the reconstructed underwater image.

8. The underwater image compression method at an extremely low bit rate according to claim 1, characterized in that: The hybrid loss function used by the foreground-assisted background generation network is expressed as: Where k′=10, and In the formula, ||·||1 represents the L1 norm, represents the GAN loss function used to optimize the generator G and the discriminator D, represents the feature matching loss function, is the kth discriminator D k The i-th layer feature extractor, n is the total number of layers, N i is the number of elements in each layer, x is the original underwater image input, c is the preset discriminator input condition, represents the perceptual loss function, V (j) is the j-th layer feature tensor of the pre-trained VGG-19 network, x′ represents the reconstructed image, m represents the number of network layers, represents the pixel-wise loss function.

9. An underwater image compression device at an extremely low bit rate, comprising a memory, a processor, and a program stored in the memory, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 8 is implemented.

10. A storage medium having a program stored thereon, characterized in that: When the program is executed, the method according to any one of claims 1 to 8 is implemented.

Images