Semantic information source coding and decoding method and system suitable for underwater image transmission

Abstract

The invention belongs to the technical field of underwater acoustic signal processing, and relates to a semantic information source coding and decoding method and system suitable for underwater image transmission. The method comprises the following steps that: a sending end inputs an original image to be transmitted into a pre-trained semantic information source coding module, sequentially extracts semantic information, quantifies the extracted semantic information of floating-point numbers, and outputs 0 / 1 bits; 0 / 1 bits are modulated, and obtained complex domain symbols are sent to an equivalent symbol domain underwater acoustic channel for transmission; a receiving end synchronizes and demodulates the received signal, and outputs 0 / 1 bits after channel equalization; and 0 / 1 bits are input into a pre-trained semantic information source decoding module, and image reconstruction is realized according to semantic information through vector removal quantization and semantic information recovery. The underwater image transmission system realizes rapid and reliable transmission of underwater images, and has the advantages of simple system structure, remarkable communication performance, high practicability and the like.

Description

Technical Field

[0001] The invention belongs to the technical field of underwater acoustic signal processing, and in particular relates to a semantic source encoding and decoding method and system suitable for underwater image transmission. Background Art

[0002] In recent years, with the deepening of human understanding of the strategic position of the ocean and the value of marine scientific research, countries around the world have increased their research on the ocean, among which the development of underwater communication technology is of great significance. Due to the high dielectric constant and conductivity of seawater, electromagnetic waves are severely attenuated in seawater, making it difficult to use for underwater long-distance information transmission. Sound waves are currently the only known information carrier that can be transmitted over long distances underwater, especially low-frequency sound waves, which can propagate over hundreds of kilometers in seawater. Underwater, compared with electromagnetic wave communication, underwater acoustic communication has the advantages of long transmission distance, high transmission rate, easy use and flexibility. Therefore, underwater acoustic communication has been widely used in underwater equipment remote control, marine environment monitoring, and underwater information warfare, playing an important role in the development of national defense, economy and science and technology.

[0003] Nowadays, the military, ocean exploration and other fields have an urgent need for reliable and fast transmission of underwater images, especially long-distance image transmission. However, the current long-distance underwater acoustic communication is difficult to meet this demand, mainly for the following two reasons: on the one hand, the available bandwidth of long-distance underwater acoustic communication is narrow and the communication rate is low, while the compression rate of traditional source coding methods such as JPEG is not high enough, and the number of bits after compression is still large, which is difficult to effectively transmit underwater; on the other hand, the underwater acoustic channel has the characteristics of long multi-path delay spread, high environmental noise and large Doppler frequency shift, which makes it very difficult to achieve reliable communication, and the traditional source coding method has poor error resistance performance and is difficult to adapt to complex and changeable underwater acoustic channels.

[0004] Therefore, in order to achieve reliable and fast transmission of images underwater, it is urgent to explore a source coding method with both high compression rate and high error resistance. In recent years, in the field of wireless communications, semantic communication systems built through deep learning networks have significant advantages in high compression rate and high error resistance, and have become a current research hotspot. However, there are few related studies underwater. Summary of the invention

[0005] The purpose of the present invention is to overcome the defects of the prior art and propose a semantic source encoding and decoding method and system suitable for underwater image transmission, so as to realize fast, reliable and long-distance transmission of images underwater.

[0006] In order to achieve the above object, the present invention proposes a semantic source encoding and decoding method suitable for underwater image transmission, comprising:

[0007] The transmitter inputs the original image to be transmitted into the pre-trained semantic source coding module, extracts the semantic information in turn, quantizes the extracted floating-point semantic information, and outputs 0 / 1 bits; modulates the 0 / 1 bits, and the obtained complex domain symbols are sent to the equivalent symbol domain underwater acoustic channel for transmission;

[0008] After the receiving end synchronizes and demodulates the received signal, it outputs 0 / 1 bits after channel equalization; the 0 / 1 bits are input into the pre-trained semantic source decoding module, and after vector quantization and semantic information recovery, the image is reconstructed according to the semantic information.

[0009] Preferably, the semantic source coding module is an encoder architecture designed based on a CNN network, comprising: two strided convolutional layers and a vector quantization layer, wherein:

[0010] The strided convolution layer has a window size of 4x4 and a stride of 2, and is used to extract semantic information from the original image.

[0011] The vector quantization layer is used to quantize the extracted semantic information of the floating point number into 0 / 1 bits.

[0012] Preferably, the modulation includes modulating the 0 / 1 bit using at least one modulation method selected from various order amplitude modulation, frequency modulation, phase modulation, orthogonal amplitude modulation and multi-carrier modulation.

[0013] Preferably, the channel equalization includes at least one equalization method selected from the group consisting of linear equalization, frequency domain equalization, time reversal and DFE equalization.

[0014] Preferably, the semantic source decoding module is a decoder architecture designed based on a CNN network, comprising: a de-vectorization layer and two transposed convolutional layers; wherein,

[0015] The de-vector quantization layer is used to perform a de-vector quantization operation on the 0 / 1 bits to restore the semantic information of the picture;

[0016] The transposed convolutional layer window size is 4x4, and the stride is 2; it is used to reconstruct the image according to the semantic information.

[0017] Preferably, the method includes the steps of training a semantic source encoding module and a semantic source decoding module; including:

[0018] A training set is established, and the Adam algorithm is used as the optimizer to train the semantic source encoding module and the semantic source decoding module respectively until the training requirements are met, thereby obtaining the trained semantic source encoding module and the semantic source decoding module.

[0019] Preferably, the strategy of the Adam algorithm is:

[0020]

[0021] Among them, g t Represents the gradient value of the loss function L to the model parameter θ, m t and v t are the first-order momentum term and the second-order momentum term, β 1 and β 2 is the exponential decay rate, with default values ​​of 0.9 and 0.999, and the default value of the regularization term ε is 1×10 -8 , and are their respective correction values, α is the learning rate, θ t ,θ t-1 are the updated parameters and the parameters solved in the previous step respectively.

[0022] On the other hand, the present invention proposes a semantic source coding system suitable for underwater image transmission, comprising: a semantic source coding module and a modulation output module deployed at the transmitting end; and a synchronous demodulation module, a channel equalization module and a semantic source decoding module deployed at the receiving end; wherein,

[0023] The semantic information source coding module is used to extract semantic information from the original image to be transmitted in sequence, and quantize the extracted semantic information of the floating point number to output 0 / 1 bits;

[0024] The modulation output module is used to modulate the 0 / 1 bit and send the obtained complex domain symbol to the equivalent symbol domain underwater acoustic channel for transmission;

[0025] The synchronous demodulation module is used to synchronize and demodulate the received signal;

[0026] The channel equalization module is used to perform channel equalization on the demodulated signal and output 0 / 1 bits;

[0027] The semantic source decoding module is used to perform vector quantization and semantic information recovery on 0 / 1 bits, and realize image reconstruction based on the semantic information.

[0028] Compared with the prior art, the advantages of the present invention are:

[0029] 1. The present invention proposes an underwater acoustic communication system architecture that combines semantic-based neural networks with traditional modulation, demodulation, and channel equalization, which can achieve fast and reliable transmission of underwater images. In the constructed system, the source code is obtained by neural network training, while modulation, demodulation, and channel equalization all use traditional non-neural network methods. The method proposed in the present invention has the advantages of simple system structure, significant communication performance, and high practicality.

[0030] 2. Different from the traditional underwater acoustic communication system, the system proposed in the present invention does not require a channel coding module and can also obtain good communication performance. In the traditional underwater acoustic communication system, a channel coding module is usually required, the purpose of which is to correct the signal and reduce the bit error rate by adding controllable redundancy, thereby improving the communication performance. However, the source coding module composed of a neural network in the present invention itself has a certain error resistance capability, and reliable transmission of underwater images can be achieved without channel coding. Therefore, the limited bandwidth resources of underwater acoustic communication can be used to transmit more useful information instead of adding redundancy, which is conducive to improving the utilization of the frequency band.

[0031] 3. Compared with traditional image coding algorithms such as JPEG, the semantic source coding method proposed in the present invention has significant advantages in compression rate and error resistance performance. Specifically, compared with traditional image coding algorithms such as JPEG that set important compression parameters through experience, the method proposed in the present invention can fully learn and extract important semantic information of the image through the neural network, thereby highly compressing the image redundancy; in addition, the neural network itself has a certain error resistance capability, which can correct the signal with errors after passing through the underwater acoustic channel, thereby improving the communication performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 It is a structure diagram of a semantic source encoding and decoding network suitable for underwater image transmission proposed by the present invention;

[0033] Figure 2 This is a comparison chart of the image compression performance between the proposed method and the traditional JPEG method;

[0034] Figure 3 is the system block diagram of the proposed method for underwater image transmission;

[0035] Figure 4 is the original picture sent in Example 2;

[0036] Figure 5 is the impulse response diagram of the deep sea water acoustic channel used in Example 2;

[0037] Figure 6 is a graph showing the variation of the bit error rate with the signal-to-noise ratio in Example 2;

[0038] Figure 7 This is a graph showing how the image restoration effect of Example 2 changes with the signal-to-noise ratio. DETAILED DESCRIPTION

[0039] The present invention proposes a semantic source encoding and decoding method suitable for underwater image transmission, which is used to realize fast and reliable transmission of images underwater. The method comprises:

[0040] Step 1) In the training phase, the image is trained based on the vector quantization variational autoencoder (VQ_VAE) model, so that the network learns to extract the important semantic information of the image and quantizes it into a 0 / 1 sequence;

[0041] This method is mainly designed based on the CNN network. The network structure is as follows Figure 1 As shown. This network adopts an encoder-decoder architecture as a whole. The encoder first extracts semantic information from the original image through a convolution layer, and then quantizes the extracted semantic information of the floating point number into 0 / 1 bits through a vector quantization layer. The present invention refers to the extraction of semantic information and vector quantization of the original image as semantic source coding. The decoder first performs a de-vectorization operation on the 0 / 1 bits to restore the semantic information of the image, and then reconstructs the image according to the semantic information through a transposed convolution layer. The present invention refers to the restoration of semantic information and vector quantization of 0 / 1 bits as semantic source decoding.

[0042] This paper uses the Adam algorithm as the optimizer for model training. The algorithm strategy can be expressed as:

[0043]

[0044] Where m t and v t are the first-order momentum term and the second-order momentum term, g t represents the gradient value of the loss function L to the model parameter θ, β 1 and β 2 is the exponential decay rate, with default values ​​of 0.9 and 0.999, and the default value of the regularization term ε is 1×10 -8 , and are their respective correction values.

[0045] Step 2) After the training is completed, the proposed semantic source encoding and decoding method is used for underwater image transmission. Figure 3 As shown. For the image to be sent, the semantic information of the original image is first extracted through semantic source coding, and then the semantic information of the extracted floating point number is quantized into 0 / 1 bits;

[0046] Step 3) The 0 / 1 bit obtained in step 2) is modulated into a complex domain symbol by any modulation method such as traditional BPSK (binary phase shift keying), QAM (quadrature amplitude modulation), etc., and then sent to the equivalent symbol domain underwater acoustic channel; it should be noted that the modulation includes modulating the 0 / 1 bit by at least one modulation method selected from various order amplitude modulation, frequency modulation, phase modulation, quadrature amplitude modulation and multi-carrier modulation. BPSK and QAM are only examples and are not limiting.

[0047] Preferably, the channel equalization includes at least one equalization method selected from linear equalization, frequency domain equalization, time reversal and DFE equalization. For example, DFE (decision feedback equalization) is only an example and is not limiting.

[0048] The signal will be distorted when it passes through the symbol domain underwater acoustic channel. The function is as follows:

[0049]

[0050] in represents convolution, x(n), y(n) and w(n) represent the transmitted signal, received signal and additive white Gaussian noise respectively, h(n) represents the impulse response of the underwater acoustic channel, and Y(k), X(k), H(k) and W(k) correspond to the frequency domain representation respectively.

[0051] At the receiving end, after completing synchronization and demodulation of the signal with a certain degree of distortion through the symbol domain underwater acoustic channel, it is necessary to first pass through any channel equalization method including DFE equalization to resist multipath interference and output 0 / 1 bits.

[0052] Step 4) The 0 / 1 bits obtained in step 3) are input into the decoding end of the neural network, and the image is reconstructed after semantic source decoding.

[0053] The technical solution of the present invention is described in detail below with reference to the accompanying drawings and embodiments.

[0054] Example 1

[0055] Embodiment 1 of the present invention proposes a semantic source coding method suitable for underwater image transmission, comprising the following steps:

[0056] (I) Training images based on the vector quantization variational autoencoder (VQ_VAE) model:

[0057] The semantic source encoding and decoding part of the present invention adopts a vector quantized variational autoencoder (VQ_VAE) model, which is an encoder-decoder architecture. The specific network structure is as follows: Figure 1 As shown. The semantic source coding part is mainly composed of two strided convolutional layers and a vector quantization layer, where the window size of the strided convolutional layer is 4x4 and the stride is 2. At the encoding end, the network first extracts the semantic information of the original image through the strided convolutional layer, and then quantizes the extracted floating-point semantic information into 0 / 1 bits through the vector quantization layer. The semantic source decoding part is mainly composed of a de-vectorization layer and two transposed convolutional layers, where the window size of the transposed convolutional layer is also 4x4 and the stride is still 2. At the decoding end, the network first de-vectorizes the 0 / 1 bits to restore the semantic information of the image, and then reconstructs the image based on the semantic information.

[0058] (II) Encode the sent image into a 0 / 1 sequence through a semantic source:

[0059] After the training is completed, for the pictures to be sent, the pictures are first encoded into 0 / 1 sequences through semantic source coding, that is, the semantic information of the original pictures is first extracted through the neural network, and then quantized into 0 / 1 bits.

[0060] (iii) The 0 / 1 sequence obtained in step (ii) is modulated into complex domain symbols by any modulation method such as BPSK, QAM, etc., and then sent to an equivalent symbol domain underwater acoustic channel. Since the signal will be distorted to a certain extent after passing through the underwater acoustic channel, at the receiving end, after completing synchronization and demodulation of the distorted signal, any equalization method including DFE is required to resist multipath interference and output 0 / 1 bits.

[0061] (iv) Input the 0 / 1 bits obtained in step (iii) into the decoding end of the network, and complete the reconstruction of the image through the semantic source decoding part.

[0062] Example 2

[0063] like Figure 1 As shown, Example 2 of the present invention uses an underwater acoustic communication system as the application background, and the effectiveness of the present invention is demonstrated through simulation verification. Specifically, it includes:

[0064] Step 1: Train the image based on the vector quantized variational autoencoder (VQ_VAE) architecture.

[0065] Specific network models and parameters are as follows Figure 1As shown: the data set selected by the network is a self-built picture set of marine vessels selected from the ImageNet data set, of which the training set has 6318 pictures and the validation set has 253 pictures. The width and height of each picture are set to 88. The number of hidden layer channels is 256, the number of residual block channels is 64, the quantization dimension is 256, the quantization codebook size is 1024, and the number of residual block layers is 4. The compression ratio set in this embodiment is 2.6%, that is, the number of bits of the original picture after semantic source coding is 2.6% of the number of bits of the original picture.

[0066] In addition, the present invention also compares the compression effect of the proposed method with that of the JPEG compression algorithm on the same image at different compression ratios, and uses the structural similarity index (SSIM) commonly used in the field of computer vision to calculate the similarity between the compressed image and the original image as a measure of the compression effect. The results are as follows: Figure 2 As shown. It can be seen that under the same compression ratio, the SSIM value of the proposed method is higher; under the same SSIM value, the compression ratio of the proposed method is lower and the compression rate is higher; therefore, the method proposed in the present invention can achieve better recovery effect under the condition of higher compression rate. The compression rate of the network model in this example is 2.6%. Under similar picture recovery effect (similar SSIM index), the compression rate of JPEG is 4.1%.

[0067] Step 2: After the training is completed, the sent picture is encoded as 0 / 1 through the semantic source. The original picture sent is as follows Figure 4 As shown, the width and height of this image are both 88, with three channels of R, G, and B. The initial size is 185856 bits, and it is 4840 bits after semantic source encoding.

[0068] Step 3: The 0 / 1 sequence obtained in step 2 is sent to the equivalent symbol domain underwater acoustic channel after traditional BPSK modulation. After synchronization and demodulation are completed at the receiving end, the 0 / 1 sequence is output after resisting multipath interference through the traditional DFE equalization method.

[0069] The underwater acoustic channel selected is the measured underwater acoustic channel, and its channel impulse response is as follows: Figure 5 As shown. After the channel convolution, the transmitted channel is additionally added with a certain amount of Gaussian noise. The signal-to-noise ratio range is [3,18] dB. The signal-to-noise ratio here is defined as:

[0070]

[0071] After the signal passes through the noisy channel, it will produce a certain amount of bit errors. Then it will be equalized by traditional DFE to reduce the bit error rate. The bit error rate after DFE equalization changes with the signal-to-noise ratio as shown in the figure below: Figure 6 shown.

[0072] Step 4: After semantic source decoding, the image is restored. Figure 7 As shown in the figure, when the signal-to-noise ratio is 8dB, the bit error rate is 1.6%. Although the image restored after semantic source decoding has certain distortion, the main content of the image can still be seen, and effective image reconstruction is achieved.

[0073] Furthermore, the image restoration results of the images compressed by JPEG under the same conditions after passing through the underwater acoustic channel are calculated. For the images sent, the proposed method uses a compression rate of 2.6%, and the JPEG method uses a compression rate of 4.1%. Figure 5 After adding noise with the same SNR, the image restoration results are compared. Figure 7 As shown in the figure, it can be seen that the JPEG method can only restore the image when the signal-to-noise ratio is higher than 15dB (0 bit error), and there is an obvious "cliff effect", that is, when there are certain bit errors and the signal-to-noise ratio is lower than 15dB, the image restoration effect will drop sharply; while the proposed method can still restore the image well when the signal-to-noise ratio is 8dB (bit error rate is 1.6%), and there is no "cliff effect". In summary, compared with the traditional JPEG compression method, the proposed semantic source coding method can have higher error resistance performance at a higher compression rate and is more suitable for underwater remote image transmission.

[0074] Example 3

[0075] Embodiment 3 of the present invention proposes a semantic source coding system suitable for underwater image transmission, which is implemented based on the method of embodiment 1 and includes: a semantic source coding module and a modulation output module deployed at the transmitting end; and a synchronous demodulation module, a channel equalization module and a semantic source decoding module deployed at the receiving end; wherein,

[0076] The semantic source coding module is used to extract semantic information from the original image to be transmitted, quantize the semantic information of the extracted floating point number, and output 0 / 1 bits;

[0077] A modulation output module is used to modulate the 0 / 1 bit and send the obtained complex domain symbol to the equivalent symbol domain underwater acoustic channel for transmission;

[0078] A synchronous demodulation module, used for synchronizing and demodulating the received signal;

[0079] A channel equalization module is used to perform channel equalization on the demodulated signal and output 0 / 1 bits;

[0080] The source decoding module is used to perform vector quantization and semantic information recovery on 0 / 1 bits, and realize image reconstruction based on the semantic information.

[0081] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention is described in detail with reference to the embodiments, it should be understood by those skilled in the art that any modification or equivalent replacement of the technical solutions of the present invention does not depart from the spirit and scope of the technical solutions of the present invention and should be included in the scope of the claims of the present invention.

Claims

1. A semantic source encoding and decoding method suitable for underwater image transmission, comprising: The transmitter inputs the original image to be transmitted into the pre-trained semantic source coding module, extracts the semantic information in turn, quantizes the extracted floating-point semantic information, and outputs 0 / 1 bits; modulates the 0 / 1 bits, and the obtained complex domain symbols are sent to the equivalent symbol domain underwater acoustic channel for transmission; After the receiving end synchronizes and demodulates the received signal, it outputs 0 / 1 bits after channel equalization; the 0 / 1 bits are input into the pre-trained semantic source decoding module, and after vector quantization and semantic information recovery, the image is reconstructed according to the semantic information.

2. The semantic source coding method for underwater image transmission according to claim 1, characterized in that: The semantic source coding module is an encoder architecture designed based on a CNN network, including two strided convolutional layers and a vector quantization layer, wherein: The strided convolution layer has a window size of 4x4 and a stride of 2, and is used to extract semantic information from the original image. The vector quantization layer is used to quantize the extracted semantic information of the floating point number into 0 / 1 bits.

3. The semantic source coding method for underwater image transmission according to claim 1, characterized in that: The modulation includes modulating the 0 / 1 bit by using at least one modulation method selected from various levels of amplitude modulation, frequency modulation, phase modulation, orthogonal amplitude modulation and multi-carrier modulation.

4. The semantic source coding method for underwater image transmission according to claim 1, characterized in that: The channel equalization includes adopting at least one equalization method among linear equalization, frequency domain equalization, time reversal and DFE equalization.

5. The semantic source coding method for underwater image transmission according to claim 1, characterized in that: The semantic source decoding module is a decoder architecture designed based on a CNN network, including: a de-vectorization layer and two transposed convolutional layers; wherein, The de-vector quantization layer is used to perform a de-vector quantization operation on the 0 / 1 bits to restore the semantic information of the picture; The transposed convolutional layer window size is 4x4, and the stride is 2; it is used to reconstruct the image according to the semantic information.

6. The semantic source coding method for underwater image transmission according to claim 1, characterized in that: The method includes the steps of training a semantic information source encoding module and a semantic information source decoding module; including: A training set is established, and the Adam algorithm is used as the optimizer to train the semantic source encoding module and the semantic source decoding module respectively until the training requirements are met, thereby obtaining the trained semantic source encoding module and the semantic source decoding module.

7. The semantic source coding method for underwater image transmission according to claim 6, characterized in that: The strategy of the Adam algorithm is: Among them, g t Represents the gradient value of the loss function L to the model parameter θ, m t and v t are the first-order momentum term and the second-order momentum term, β1 and β2 are exponential decay rates, with default values ​​of 0.9 and 0.999, respectively. The default value of the regularization term ε is 1×10 -8 , and are their respective correction values, α is the learning rate, θ t ,θ t-1 are the updated parameters and the parameters solved in the previous step respectively.

8. A semantic source coding system suitable for underwater image transmission, characterized in that: include: The semantic source coding module and modulation output module deployed at the transmitter; And the synchronous demodulation module, channel equalization module and semantic source decoding module deployed at the receiving end; among them, The semantic information source coding module is used to extract semantic information from the original image to be transmitted in sequence, and quantize the extracted semantic information of the floating point number to output 0 / 1 bits; The modulation output module is used to modulate the 0 / 1 bit and send the obtained complex domain symbol to the equivalent symbol domain underwater acoustic channel for transmission; The synchronous demodulation module is used to synchronize and demodulate the received signal; The channel equalization module is used to perform channel equalization on the demodulated signal and output 0 / 1 bits; The semantic source decoding module is used to perform vector quantization and semantic information recovery on 0 / 1 bits, and realize image reconstruction based on the semantic information.

Images