Underwater image transmission method based on underwater sound semantic communication

Abstract

The invention discloses an underwater image transmission method based on underwater sound semantic communication, and belongs to the technical field of image transmission. Comprising the following steps: acquiring an underwater image, and converting the underwater image into a corresponding description text; extracting entities in the description text to form an entity list, and taking the entity list as a text semantic representation form of the underwater image; transmitting the entity list through underwater acoustic communication; performing semantic detection and correction on the received entity list to obtain an accurate entity list; converting the accurate entity list into a description text; and converting the description text into an underwater image which is consistent with the semantics of the original image. According to the method, the underwater image is converted into text semantics through the trace expression characteristic of semantic communication, so that the data transmission quantity is remarkably reduced, and the requirement on the underwater acoustic channel bandwidth is reduced; according to the method, the robustness of semantic communication is utilized, the technical advantages of deep learning and the like are combined, accurate information transmission and semantic integrity are ensured under the low signal-to-noise ratio of an underwater acoustic channel, and the reconstruction and anti-interference performance in underwater image transmission is improved.

Description

Technical Field

[0001] The invention discloses an underwater image transmission method based on underwater acoustic semantic communication, belonging to the technical field of image transmission. Background Art

[0002] Underwater acoustic communication is the main means of transmitting information underwater using sound waves. It is currently the key technology for achieving medium and long-distance wireless communication between underwater targets and has been widely used in underwater detection, marine resource development, seabed scientific research and other fields. However, the characteristics of underwater acoustic channels are limited bandwidth resources and complex environmental noise, which leads to severe challenges for traditional underwater acoustic communication in terms of efficient and reliable data transmission. Specifically, limited bandwidth resources limit the efficient transmission of data, while complex environmental noise affects the reliable transmission of data, which seriously restricts the application scope and performance of underwater acoustic communication. Semantic communication is an emerging communication technology whose core idea is to convey the meaning of data rather than the data itself. This technology reduces the bandwidth requirement of underwater acoustic channels by compressing a large amount of raw data into a small amount of semantic information, thereby achieving efficient data transmission. At the same time, semantic communication reconstructs information through deep learning technology and semantic knowledge base, enhances anti-interference ability, and ensures the reliability of data transmission. Therefore, it is an important scientific problem that needs to be solved to combine the advantages of semantic communication with underwater acoustic communication and develop an innovative underwater acoustic communication method to overcome the narrow bandwidth and low noise limitations of underwater acoustic channels. Summary of the invention

[0003] The purpose of the present invention is to provide an underwater image transmission method based on underwater acoustic semantic communication to solve the problem in the prior art that the narrow bandwidth and low signal-to-noise ratio of the underwater acoustic channel lead to insufficient underwater image transmission performance.

[0004] An underwater image transmission method based on underwater acoustic semantic communication comprises acquiring an underwater image and converting the underwater image into corresponding description text; extracting entities in the description text to form an entity list as a text semantic representation form of the underwater image; transmitting the entity list through underwater acoustic communication; performing semantic detection and correction on the received entity list to acquire an accurate entity list; converting the accurate entity list into description text, and converting the description text into an underwater image having semantic consistency with the original image.

[0005] An underwater image description model is trained based on an underwater image dataset with text annotations, and the underwater images are converted into corresponding description texts through the underwater image description model.

[0006] The underwater image description model is a CNN-LSTM, which includes inputting the underwater image into an image encoder to obtain image features, and inputting the image features into a text decoder to obtain description text. The image encoder includes multiple convolutional neural networks CNN, and the text decoder includes an attention mechanism and a long short-term memory neural network LSTM.

[0007] Preferably, the underwater image description model is a CNN-RNN, and the CNN-RNN includes inputting the underwater image into an image encoder to obtain image features, and inputting the image features into a text decoder to obtain description text. The image encoder includes multiple convolutional neural networks CNN, and the text decoder includes multiple recurrent neural networks RNN.

[0008] Preferably, the underwater image description model is a CNN-Transformer, and the CNN-Transformer includes inputting the underwater image into an image encoder to obtain image features, and inputting the image features into a text decoder to obtain description text. The image encoder includes multiple convolutional neural networks CNN, and the text decoder is a Transformer module.

[0009] An entity recognition model is trained based on a text dataset with entity annotations, and entities in the description text are extracted through the entity recognition model to form an entity list.

[0010] The entity recognition model inputs the description text into the BERT-BiLSTM-CRF module, the BERT-BiLSTM-CRF module generates entities and inputs them into the entity classification module, the entity classification module assigns a category label to each entity, and then outputs an entity list. The BERT-BiLSTM-CRF module includes a pre-trained language model module BERT, a bidirectional long short-term memory network module BiLSTM, and a conditional random field module CRF.

[0011] Preferably, the entity recognition model inputs the description text into a BiLSTM-CRF module, the BiLSTM-CRF module generates entities and inputs them into an entity classification module, the entity classification module assigns a category label to each entity, and then outputs an entity list, and the BiLSTM-CRF module includes a bidirectional long short-term memory network module BiLSTM and a conditional random field module CRF.

[0012] Preferably, the entity recognition model inputs the description text into an RNN-CRF module, the RNN-CRF module generates entities and inputs them into an entity classification module, the entity classification module assigns a category label to each entity, and then outputs an entity list, and the RNN-CRF module includes a recurrent neural network module RNN and a conditional random field module CRF.

[0013] Preferably, the entity recognition model inputs the description text into a RoBERTa module, the RoBERTa module generates entities and inputs them into an entity classification module, the entity classification module assigns a category label to each entity, and then outputs an entity list.

[0014] The underwater acoustic communication includes coding and modulation, underwater acoustic channel and decoding and demodulation;

[0015] The encoding and modulation convert the entity list into a bit stream adapted to the transmission through the underwater acoustic channel, and the decoding and demodulation restore the bit stream passing through the underwater acoustic channel to the entity list.

[0016] Through the semantic detection and correction mechanism, the transmitted entity list is processed in combination with the semantic knowledge base to generate a detected and corrected entity list.

[0017] A text generation model is trained based on the text dataset, and the accurate entity list is converted into descriptive text through the text generation model.

[0018] The text generation model inputs the entity list into the draft construction module to generate a draft, and then inputs the draft into the BART module to generate a description text.

[0019] Preferably, the text generation model inputs the entity list into a draft construction module to generate a draft, and then inputs the draft into a BERT module to generate a description text.

[0020] Preferably, the text generation model inputs the entity list into the draft construction module to generate a draft, and then inputs the draft into the T5 module to generate the description text.

[0021] A text-generated image model is trained based on an underwater image dataset with text annotations. The text-generated image model is used to convert the description text into an underwater image with semantics consistent with the original image.

[0022] The text-generated image model is a diffusion model, which includes inputting the description text into a CLIP text encoder to generate text features, and inputting the text features and a noise image together into an image generator to generate an underwater image. The image generator includes a U-Net network and a scheduling algorithm module and a variational autoencoder VAE.

[0023] Preferably, the text-to-image model is a StackGAN model, the description text generates text features through a text encoder, and then inputs the text into an image decoder to generate an underwater image, the text encoder is a Transformer module, and the image decoder is a conditional generative adversarial network CGAN.

[0024] Preferably, the text-generated image model is a Parti model, which includes extracting text features describing the text through a text encoder, and then converting the text features into an underwater image through an image generator, the text encoder is a T5 module, and the image generator includes an autoregressive Transformer module and a VQ-VAE module.

[0025] Compared with the prior art, the present invention has the following beneficial effects: the present invention converts underwater images into trace text semantics for transmission through the characteristics of trace expressions in semantic communication, which significantly reduces the amount of data transmission and effectively alleviates the problem of limited bandwidth of the underwater acoustic channel, thereby reducing the demand for channel bandwidth. At the same time, by utilizing the robustness of semantic communication and combining the advantages of deep learning technology and semantic knowledge base, the accurate transmission of information and semantic integrity can be ensured even in complex environments with low signal-to-noise ratio, significantly improving the reconstruction capability and anti-interference performance of underwater images. In general, the present invention innovatively combines semantic communication with underwater acoustic communication, provides a new technical route for underwater image transmission, and provides an effective solution to the technical problems of narrow bandwidth and low signal-to-noise ratio of underwater acoustic channels. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 This is a flow chart of an underwater image transmission method based on underwater acoustic semantic communication according to the present invention;

[0027] Figure 2 A structural diagram of the model for underwater image description;

[0028] Figure 3 This is the structural diagram of the entity recognition model;

[0029] Figure 4 A structural diagram of the model for text generation;

[0030] Figure 5 A structural diagram of the image model generated for text;

[0031] Figure 6 The original underwater image and the underwater images generated by different image coding methods when the signal-to-noise ratio is 15;

[0032] Figure 7 The original underwater image and the underwater images generated by different image coding methods when the signal-to-noise ratio is 3. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention is described clearly and completely below. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0034] Embodiment 1:

[0035] An underwater image transmission method based on underwater acoustic semantic communication comprises acquiring an underwater image and converting the underwater image into corresponding description text; extracting entities in the description text to form an entity list as a text semantic representation form of the underwater image; transmitting the entity list through underwater acoustic communication; performing semantic detection and correction on the received entity list to acquire an accurate entity list; converting the accurate entity list into description text, and converting the description text into an underwater image having semantic consistency with the original image.

[0036] An underwater image description model is trained based on an underwater image dataset with text annotations, and the underwater images are converted into corresponding description texts through the underwater image description model.

[0037] The underwater image description model is a CNN-LSTM, which includes inputting the underwater image into an image encoder to obtain image features, and inputting the image features into a text decoder to obtain description text. The image encoder includes multiple convolutional neural networks CNN, and the text decoder includes an attention mechanism and a long short-term memory neural network LSTM.

[0038] An entity recognition model is trained based on a text dataset with entity annotations, and entities in the description text are extracted through the entity recognition model to form an entity list.

[0039] The entity recognition model inputs the description text into the BERT-BiLSTM-CRF module, the BERT-BiLSTM-CRF module generates entities and inputs them into the entity classification module, the entity classification module assigns a category label to each entity, and then outputs an entity list. The BERT-BiLSTM-CRF module includes a pre-trained language model module BERT, a bidirectional long short-term memory network module BiLSTM, and a conditional random field module CRF.

[0040] The underwater acoustic communication includes coding and modulation, underwater acoustic channel and decoding and demodulation;

[0041] The encoding and modulation convert the entity list into a bit stream adapted to the transmission through the underwater acoustic channel, and the decoding and demodulation restore the bit stream passing through the underwater acoustic channel to the entity list.

[0042] Through the semantic detection and correction mechanism, the transmitted entity list is processed in combination with the semantic knowledge base to generate a detected and corrected entity list.

[0043] A text generation model is trained based on the text dataset, and the accurate entity list is converted into descriptive text through the text generation model.

[0044] The text generation model inputs the entity list into the draft construction module to generate a draft, and then inputs the draft into the BART module to generate a description text.

[0045] A text-generated image model is trained based on an underwater image dataset with text annotations. The text-generated image model is used to convert the description text into an underwater image with semantics consistent with the original image.

[0046] The text-generated image model is a diffusion model, which includes inputting the description text into a CLIP text encoder to generate text features, and inputting the text features and a noise image together into an image generator to generate an underwater image. The image generator includes a U-Net network and a scheduling algorithm module and a variational autoencoder VAE.

[0047] Embodiment 2:

[0048] The same parts as those in the first embodiment are not described in detail. The second embodiment makes improvements on the underwater image description model, entity recognition model, text generation model, and text generation image model.

[0049] The underwater image description model is a CNN-RNN, and the CNN-RNN includes inputting the underwater image into an image encoder to obtain image features, and inputting the image features into a text decoder to obtain description text. The image encoder includes multiple convolutional neural networks CNN, and the text decoder includes multiple recurrent neural networks RNN.

[0050] The entity recognition model inputs the description text into the BiLSTM-CRF module, the BiLSTM-CRF module generates entities and inputs them into the entity classification module, the entity classification module assigns a category label to each entity, and then outputs an entity list. The BiLSTM-CRF module includes a bidirectional long short-term memory network module BiLSTM and a conditional random field module CRF.

[0051] The text generation model inputs the entity list into the draft construction module to generate a draft, and then inputs the draft into the BERT module to generate a description text.

[0052] The text-to-image model is a StackGAN model, which describes the text and generates text features through a text encoder, and then inputs the text into an image decoder to generate an underwater image. The text encoder is a Transformer module, and the image decoder is a conditional generative adversarial network (CGAN).

[0053] Embodiment three:

[0054] The same parts as those in Example 1 will not be repeated here. Example 3 improves the underwater image description model, entity recognition model, text generation model, and text generation image model.

[0055] The underwater image description model is a CNN-Transformer, which includes inputting the underwater image into an image encoder to obtain image features, and inputting the image features into a text decoder to obtain description text. The image encoder includes multiple convolutional neural networks CNN, and the text decoder is a Transformer module.

[0056] The entity recognition model inputs the description text into the RNN-CRF module, the RNN-CRF module generates entities and inputs them into the entity classification module, the entity classification module assigns a category label to each entity and then outputs an entity list. The RNN-CRF module includes a recurrent neural network module RNN and a conditional random field module CRF.

[0057] The text generation model inputs the entity list into the draft construction module to generate a draft, and then inputs the draft into the T5 module to generate a description text.

[0058] The text-to-image model is a Parti model, which includes extracting text features describing the text through a text encoder, and then converting the text features into an underwater image through an image generator. The text encoder is a T5 module, and the image generator includes an autoregressive Transformer module and a VQ-VAE module.

[0059] Embodiment 4:

[0060] The same parts as those in the first embodiment are not described in detail. The fourth embodiment improves the entity recognition model.

[0061] The entity recognition model inputs the description text into the RoBERTa module, the RoBERTa module generates entities and inputs them into the entity classification module, the entity classification module assigns a category label to each entity, and then outputs an entity list.

[0062] In the present invention, a method for underwater image transmission based on underwater acoustic semantic communication is provided as follows: Figure 1As shown, the method includes acquiring an underwater image and converting the underwater image into corresponding description text; extracting entities in the description text to form an entity list as a text semantic representation form of the underwater image; transmitting the entity list through underwater acoustic communication, performing semantic detection and correction on the received entity list, and obtaining an accurate entity list; converting the accurate entity list into description text, and converting the description text into an underwater image that is semantically consistent with the original image.

[0063] The structure diagram of the underwater image description model is as follows Figure 2 As shown, the present invention is based on the underwater image dataset with text annotations to obtain an underwater image description model CNN-LSTM, and the underwater image is converted into corresponding description text through CNN-LSTM. The CNN-LSTM includes inputting the underwater image into an image encoder to obtain image features, and inputting the image features into a text decoder to obtain description text. The image encoder includes multiple convolutional neural networks CNN, and the text decoder includes an attention mechanism and a long short-term memory neural network LSTM.

[0064] The structure diagram of the entity recognition model is as follows Figure 3 As shown, the present invention obtains an entity recognition model based on a text data set with entity annotations, and extracts entities in the description text through the entity recognition model to form an entity list. The entity recognition model inputs the description text into a BERT-BiLSTM-CRF module, the BERT-BiLSTM-CRF module generates entities and inputs them into an entity classification module, the entity classification module assigns a category label to each entity, and then outputs an entity list, and the BERT-BiLSTM-CRF module includes a pre-trained language model module BERT, a bidirectional long short-term memory network module BiLSTM, and a conditional random field module CRF.

[0065] The structure diagram of the text generation model is as follows Figure 4 As shown, the present invention obtains a text generation model based on text data set training, and converts an accurate entity list into a description text through the text generation model. The text generation model inputs the entity list into a draft construction module to generate a draft, and then inputs the draft into a BART module to generate a description text. The BART module includes a bidirectional encoder module and an autoregressive decoder module.

[0066] The structure diagram of the text generation image model is as follows Figure 5As shown, the present invention obtains a text generation image model based on an underwater image dataset with text annotations, and converts the description text into an underwater image with semantics consistent with the original image through the text generation image model. The text generation image model is a diffusion model, including inputting the description text into a CLIP text encoder to generate text features, and inputting the text features and the noise image together into an image generator to generate an underwater image. The image generator includes a U-Net network, a scheduling algorithm module, and a variational autoencoder VAE.

[0067] The experimental process is introduced below.

[0068] In the embodiment of the present invention, the underwater images in the underwater image dataset are first annotated with text to construct an underwater image dataset with text annotations. The dataset is derived from an underwater image collection on the Internet, including 3,176 representative underwater images, covering a variety of typical underwater scenes, such as coral reefs, underwater creatures, etc. In order to ensure the diversity and semantic integrity of the description text, each image contains 5 English description texts, which accurately describe the semantic content and scene characteristics of the image.

[0069] After obtaining an underwater image dataset with text annotations, the underwater image description model is trained using this dataset so that the model can learn the mapping relationship between underwater images and corresponding description texts, thereby constructing an underwater image description model that can adapt to the image content of different underwater scenes and achieve efficient conversion from underwater images to description texts.

[0070] The embodiment of the present invention converts the acquired underwater image into the corresponding description text based on the trained underwater image description model, specifically: the acquired underwater image is converted into the corresponding description text through the trained CNN-LSTM model. In the embodiment of the present invention, the CNN-LSTM model includes an image encoder and a text decoder. Among them, the image encoder processes the underwater image through a multi-layer convolutional neural network (CNN) to extract a high-level image feature representation; and the text decoder uses a long short-term memory network (LSTM) with an integrated attention mechanism to gradually decode these image features into semantically coherent description text. The attention mechanism can dynamically focus on different areas of image features, thereby effectively generating description text that is highly matched with the underwater image content.

[0071] The embodiment of the present invention converts the acquired underwater image into the corresponding description text based on the trained underwater image description model, specifically: the acquired underwater image is converted into the corresponding description text through the trained CNN-RNN model. In the embodiment of the present invention, the CNN-RNN model includes an image encoder and a text decoder. Among them, the image encoder processes the underwater image through a multi-layer CNN to extract a high-level image feature representation; and the text decoder uses a multi-layer recurrent neural network (RNN) to gradually decode these image features into description text.

[0072] The embodiment of the present invention converts the acquired underwater image into the corresponding description text based on the trained underwater image description model, specifically: the acquired underwater image is converted into the corresponding description text through the trained CNN-Transformer model. In the embodiment of the present invention, the CNN-Transformer model includes an image encoder and a text decoder. Among them, the image encoder processes the underwater image through a multi-layer CNN to extract a high-level image feature representation; and the text decoder encodes the image features through the Transformer model to generate a context representation, and generates a description text according to the context representation and step-by-step decoding process.

[0073] In an embodiment of the present invention, based on the description text in the underwater image data set with text annotation, the BIO annotation method is used to annotate the entity indicator for each word, thereby obtaining a text data set with entity annotation. The BIO annotation method marks each word as "B-entity" (the starting part of the entity), "I-entity" (the internal part of the entity) or "O" (non-entity), thereby forming a text data set with detailed entity annotation. After obtaining the text data set with entity annotation, the entity recognition model is trained through this data set, so that the model learns the mapping relationship between the description text and the corresponding entity list, thereby constructing an entity recognition model that can convert the description text into an entity list.

[0074] The embodiment of the present invention is based on a trained entity recognition model to identify entities in a description text and form an entity list, specifically: through a trained BERT-BiLSTM-CRF model and an entity classification algorithm, entities in the description text are identified to form an entity list. In the embodiment of the present invention, the entity recognition model first identifies entities from the description text through a BERT-BiLSTM-CRF model. Specifically, the BERT model processes the word sequence of the description text and converts it into a word embedding sequence with rich contextual semantic information. The sequence is then input into a bidirectional long short-term memory network (BiLSTM). BiLSTM models the bidirectional dependency of the sequence and generates a feature vector sequence. The conditional random field (CRF) selects the optimal entity labeling sequence from all possible entity labeling sequences through the feature vector sequence and the dependency of the label sequence, thereby efficiently identifying the entities in the description text. The identified entities are presented in the form of words or phrases, and each entity is assigned a category label through an entity classification algorithm to generate an entity list. Each entry in the entity list contains the entity and its corresponding category label. For example, in the entry "pipeline[SED]", "pipeline" represents the entity and "SED" represents its category label. In addition, semantic information such as entities and their category labels are important components of the semantic knowledge base. In the semantic knowledge base, this information can be used to assist in resolving semantic ambiguity problems that may occur during the transmission process, thereby further improving the accuracy of underwater image transmission.

[0075] The embodiment of the present invention identifies entities in the description text based on the trained entity recognition model to form an entity list, specifically: through the trained BiLSTM-CRF model and entity classification algorithm, the entities in the description text are identified to form an entity list. In the embodiment of the present invention, the entity recognition model first identifies entities from the description text through the BiLSTM-CRF model. Specifically, the context representation of each word in the description text is obtained through BiLSTM to generate a context-related hidden state vector. CRF uses this vector, combined with the dependency of the label sequence, to select the optimal entity annotation sequence from all possible entity annotation sequences to identify the entities in the description text. The identified entities are presented in the form of words or phrases, and each entity is assigned a category label through the entity classification algorithm to generate an entity list.

[0076] The embodiment of the present invention identifies entities in the description text based on the trained entity recognition model and forms an entity list. Specifically, the entities in the description text are identified through the trained RNN-CRF model and entity classification algorithm to form an entity list. In the embodiment of the present invention, the entity recognition model first identifies entities from the description text through the RNN-CRF model. Specifically, the sequence of the description text is contextually modeled by RNN to generate a context representation of each word to form a context representation sequence. CRF uses this sequence, combined with the dependency of the label sequence, to select the optimal entity annotation sequence from all possible entity annotation sequences to identify the entities in the description text. The identified entities are presented in the form of words or phrases, and each entity is assigned a category label through the entity classification algorithm to generate an entity list.

[0077] The embodiment of the present invention identifies entities in the description text based on the trained entity recognition model to form an entity list, specifically: through the trained RoBERTa model and entity classification algorithm, the entities in the description text are identified to form an entity list. In the embodiment of the present invention, the entity recognition model first extracts the contextual representation of each word in the description text through the RoBERTa encoder, and predicts whether each word is an entity through the classifier. The predicted entity annotation sequence is then mapped back to the entity to identify the entity in the description text. The identified entities are presented in the form of words or phrases, and each entity is assigned a category label through the entity classification algorithm to generate an entity list.

[0078] The present invention processes the entity list after transmission through a semantic detection and correction mechanism to generate an accurate entity list. In an embodiment of the present invention, the received entity list is first preliminarily processed through a semantic detection algorithm to detect possible erroneous entities. Then, these erroneous entities are corrected in combination with the information in the semantic knowledge base, thereby achieving accurate information transmission. Specifically, each entity will be evaluated based on its category label to determine whether the category label is consistent with the entity category. If consistent, the entity is confirmed to be accurate; if inconsistent, it needs to be corrected, and a corrected entity is generated based on the category label. The semantic knowledge base plays an important role in this process, providing rich contextual information, so that the correction of entities can be accurately judged based on the context and the relationship between entities, thereby ensuring the reliability of information transmission.

[0079] The embodiment of the present invention converts the entity list into a descriptive text based on a trained text generation model, specifically: the entity list is converted into a descriptive text through a draft construction algorithm and a trained BART model. In the embodiment of the present invention, the text generation model first converts the entity list into a more understandable draft through a draft construction algorithm. The draft construction algorithm aims to capture the relationship between entities and their semantic meanings, and organize them in a concise and clear way to provide a clear framework for subsequent text generation. Then, through the BART model, the generated draft is used as input to generate a descriptive text through a bidirectional encoder and an autoregressive decoder. Among them, the bidirectional encoder can capture more comprehensive contextual information through a bidirectional sequence modeling method, thereby improving the accuracy of text generation, and the autoregressive decoder gradually derives the generation process of each word when generating text content to ensure the coherence and rationality of the text.

[0080] The embodiment of the present invention converts the entity list into a description text based on a trained text generation model, specifically: the entity list is converted into a description text through a draft construction algorithm and a trained BERT model. In the embodiment of the present invention, the text generation model first converts the entity list into a draft through a draft construction algorithm. Then, the generated draft is used as input to generate a description text through the BERT model. The BERT model comprehensively considers the relationship between each word in the draft and the context through a bidirectional self-attention mechanism, thereby generating a fluent description text.

[0081] The embodiment of the present invention converts the entity list into descriptive text based on the trained text generation model, specifically: the entity list is converted into descriptive text through the draft construction algorithm and the trained T5 model. In the embodiment of the present invention, the text generation model first converts the entity list into a draft through the draft construction algorithm. Then, the generated draft is used as input to generate the descriptive text through the T5 model. When processing the draft, the T5 model first encodes the information in the draft into a high-dimensional vector representation to capture the contextual relationships and detailed features in the text. Then, the T5 model uses its powerful autoregressive generation capability to gradually generate a fluent and coherent descriptive text from the encoded vector representation.

[0082] The embodiment of the present invention converts the description text into an underwater image that is semantically consistent with the original image based on the trained text-generated image model, specifically: the description text is converted into an underwater image that is semantically consistent with the original image through the trained diffusion model. In the embodiment of the present invention, the text-generated image model includes a CLIP text encoder and an image generator. The core of the model is the diffusion model. First, the CLIP text encoder encodes the description text into text features, which can accurately capture the semantic information of the text and provide effective guidance for subsequent image generation. In the image generator, a random noise image is generated in the initial stage, and the random noise image is input into the U-Net network together with the text features. The U-Net network gradually optimizes the image features through an iterative denoising process, and dynamically adjusts the denoising steps through a scheduling algorithm, thereby gradually generating image features that are semantically consistent with the description text. Finally, the generated image features are decoded by a VAE decoder and converted into a pixel-level underwater image. The generated underwater image can accurately retain the semantic information of the original description text and ensure semantic consistency.

[0083] The embodiment of the present invention converts the description text into an underwater image that is semantically consistent with the original image based on a trained text-generated image model. Specifically, the description text is converted into an underwater image that is semantically consistent with the original image through a trained StackGAN model. In an embodiment of the present invention, the text-generated image model includes a text encoder and an image decoder. The core of the model is the StackGAN model. First, the description text is encoded into text features through a Transformer model. In the image decoder, a two-stage generation strategy of a conditional generative adversarial network (CGAN) is used: in the first stage, text features are converted into a rough low-resolution underwater image; in the second stage, a high-resolution image is further generated based on the low-resolution image, and finally a high-quality, detail-rich underwater image that is semantically consistent with the description text is generated.

[0084] The embodiment of the present invention converts the description text into an underwater image that is semantically consistent with the original image based on a trained text-to-image model. Specifically, the description text is converted into an underwater image that is semantically consistent with the original image through a trained Parti model. In an embodiment of the present invention, the text-to-image model includes a text encoder and an image generator. The core of the model is the Parti model. First, the description text is encoded into text features through the T5 model, which retains the global semantic information of the description text. In the image generator, a discrete image Token sequence is gradually generated through an autoregressive Transformer decoder. Finally, the Token sequence is reconstructed into a pixel-level underwater image through a VQ-VAE decoder to ensure that the underwater image has high-quality visual expression.

[0085] The following are the experimental results obtained through the experiment of Example 1.

[0086] In order to verify the feasibility of underwater acoustic semantic communication for underwater image transmission, the method proposed in this invention is compared with four standard methods. In the experiment, the four compared methods respectively use JPEG, JPEG2000, WebP and BPG for source encoding / decoding, convolutional code for channel encoding / decoding, and orthogonal frequency division multiplexing technology for modulation / demodulation, while the method proposed in this invention uses source encoding / decoding based on entity list, and other settings are consistent with the four compared methods.

[0087] Figure 6 The original underwater image and the underwater images generated by different image coding methods when the signal-to-noise ratio is 15. As can be seen from the figure, the JPEG and BPG methods in the standard method have obvious data loss. The underwater images generated by the JPEG2000 and WebP methods are consistent with the original underwater images. At the same time, the method proposed in the present invention generates clear underwater images at a high signal-to-noise ratio and can maintain semantic consistency well. In addition, the method of the present invention performs well in appearance consistency, such as color and species characteristics, which is demonstrated by the underwater images generated from Image-2.

[0088] Figure 7 The original underwater image and underwater images generated by different image coding methods when the signal-to-noise ratio is 3. Due to decoding errors, the JPEG and BPG methods in the standard method cannot generate underwater images, showing poor robustness to low signal-to-noise ratio environments. In contrast, although the JPEG2000 and WebP methods successfully generated underwater images, they both showed varying degrees of distortion, among which the WebP method had a lighter degree of distortion. The method proposed in the present invention shows significant advantages under low signal-to-noise ratios, generates the clearest underwater images, and maintains semantic consistency with the original underwater images.

[0089] Table 1 shows the evaluation index values ​​of the images generated by Image-3 under different image coding methods when the signal-to-noise ratio is 15. The evaluation indexes are image quality evaluation index and semantic similarity evaluation index, among which the image quality evaluation index includes image space quality evaluator (BRISQUE), natural image quality evaluator (NIQE), perception-based image quality evaluator (PIQE), hyper-network-based image quality assessment (HyperIQA), multi-dimensional attention network for image quality assessment (MANIQA) and Transformer model for no-reference image quality assessment (TRES), and the semantic similarity evaluation index (SSC) is obtained by the following steps: first, the underwater image is converted into description text through the underwater image description model; then, the description text is converted into a vector through the BERT model; finally, the cosine similarity between the two vectors is calculated to obtain the semantic similarity score. It can be seen from Table 1 that the method proposed in the present invention performs well in most evaluation indicators. This shows that the method proposed in the present invention can still maintain a strong advantage under high signal-to-noise ratio, ensuring that the generated underwater image can be highly consistent with human visual understanding.

[0090] Table 1 Evaluation index values ​​of images generated by Image-3 under different image coding methods when the signal-to-noise ratio is 15

[0091]

[0092] Table 2 shows the values ​​of the evaluation indexes of the images generated by Image-3 under different image coding methods when the signal-to-noise ratio is 3. It can be seen from the table that the method proposed in the present invention shows excellent performance in all evaluation indicators. This shows that compared with the standard method, the method proposed in the present invention can generate underwater images with higher quality and stronger semantic consistency under low signal-to-noise ratio.

[0093] Table 2 Evaluation index values ​​of images generated by Image-3 under different image coding methods when the signal-to-noise ratio is 3

[0094]

[0095] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some or all of the technical features may be replaced by equivalents, and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An underwater image transmission method based on underwater acoustic semantic communication, characterized in that: The method includes acquiring underwater images and converting the underwater images into corresponding description texts; extracting entities in the description texts to form an entity list as a textual semantic representation of the underwater images; transmitting the entity list through underwater acoustic communication; performing semantic detection and correction on the received entity list to obtain an accurate entity list; converting the accurate entity list into description texts, and converting the description texts into underwater images that are semantically consistent with the original images.

2. The underwater image transmission method based on underwater acoustic semantic communication according to claim 1 is characterized in that: An underwater image description model is trained based on an underwater image dataset with text annotations, and the underwater images are converted into corresponding description texts through the underwater image description model.

3. The underwater image transmission method based on underwater acoustic semantic communication according to claim 2 is characterized in that: The underwater image description model is a CNN-LSTM, which includes inputting the underwater image into an image encoder to obtain image features, and inputting the image features into a text decoder to obtain description text. The image encoder includes multiple convolutional neural networks CNN, and the text decoder includes an attention mechanism and a long short-term memory neural network LSTM.

4. The underwater image transmission method based on underwater acoustic semantic communication according to claim 2 is characterized in that: The underwater image description model is a CNN-RNN, and the CNN-RNN includes inputting the underwater image into an image encoder to obtain image features, and inputting the image features into a text decoder to obtain description text. The image encoder includes multiple convolutional neural networks CNN, and the text decoder includes multiple recurrent neural networks RNN.

5. The underwater image transmission method based on underwater acoustic semantic communication according to claim 2 is characterized in that: The underwater image description model is a CNN-Transformer, which includes inputting the underwater image into an image encoder to obtain image features, and inputting the image features into a text decoder to obtain description text. The image encoder includes multiple convolutional neural networks CNN, and the text decoder is a Transformer module.

6. The underwater image transmission method based on underwater acoustic semantic communication according to claim 1 is characterized in that: An entity recognition model is trained based on a text dataset with entity annotations, and entities in the description text are extracted through the entity recognition model to form an entity list.

7. The underwater image transmission method based on underwater acoustic semantic communication according to claim 6 is characterized in that: The entity recognition model inputs the description text into the BERT-BiLSTM-CRF module, the BERT-BiLSTM-CRF module generates entities and inputs them into the entity classification module, the entity classification module assigns a category label to each entity, and then outputs an entity list. The BERT-BiLSTM-CRF module includes a pre-trained language model module BERT, a bidirectional long short-term memory network module BiLSTM, and a conditional random field module CRF.

8. The underwater image transmission method based on underwater acoustic semantic communication according to claim 6 is characterized in that: The entity recognition model inputs the description text into the BiLSTM-CRF module, the BiLSTM-CRF module generates entities and inputs them into the entity classification module, the entity classification module assigns a category label to each entity, and then outputs an entity list. The BiLSTM-CRF module includes a bidirectional long short-term memory network module BiLSTM and a conditional random field module CRF.

9. The underwater image transmission method based on underwater acoustic semantic communication according to claim 6, characterized in that: The entity recognition model inputs the description text into the RNN-CRF module, the RNN-CRF module generates entities and inputs them into the entity classification module, the entity classification module assigns a category label to each entity and then outputs an entity list. The RNN-CRF module includes a recurrent neural network module RNN and a conditional random field module CRF.

10. The underwater image transmission method based on underwater acoustic semantic communication according to claim 6, characterized in that: The entity recognition model inputs the description text into the RoBERTa module, the RoBERTa module generates entities and inputs them into the entity classification module, the entity classification module assigns a category label to each entity, and then outputs an entity list.

11. The underwater image transmission method based on underwater acoustic semantic communication according to claim 1, characterized in that: Through the semantic detection and correction mechanism, the transmitted entity list is processed in combination with the semantic knowledge base to generate a detected and corrected entity list.

12. The underwater image transmission method based on underwater acoustic semantic communication according to claim 1, characterized in that: A text generation model is trained based on the text dataset, and the accurate entity list is converted into descriptive text through the text generation model.

13. The underwater image transmission method based on underwater acoustic semantic communication according to claim 12, characterized in that: The text generation model inputs the entity list into the draft construction module to generate a draft, and then inputs the draft into the BART module to generate a description text.

14. The underwater image transmission method based on underwater acoustic semantic communication according to claim 12, characterized in that: The text generation model inputs the entity list into the draft construction module to generate a draft, and then inputs the draft into the BERT module to generate a description text.

15. The underwater image transmission method based on underwater acoustic semantic communication according to claim 12, characterized in that: The text generation model inputs the entity list into the draft construction module to generate a draft, and then inputs the draft into the T5 module to generate a description text.

16. The underwater image transmission method based on underwater acoustic semantic communication according to claim 1, characterized in that: A text-generated image model is trained based on an underwater image dataset with text annotations. The text-generated image model is used to convert the description text into an underwater image with semantics consistent with the original image.

17. The underwater image transmission method based on underwater acoustic semantic communication according to claim 16, characterized in that: The text-generated image model is a diffusion model, which includes inputting the description text into a CLIP text encoder to generate text features, and inputting the text features and a noise image together into an image generator to generate an underwater image. The image generator includes a U-Net network and a scheduling algorithm module and a variational autoencoder VAE.

18. The underwater image transmission method based on underwater acoustic semantic communication according to claim 16, characterized in that: The text-to-image model is a StackGAN model, which describes the text and generates text features through a text encoder, and then inputs the text into an image decoder to generate an underwater image. The text encoder is a Transformer module, and the image decoder is a conditional generative adversarial network (CGAN).

19. The underwater image transmission method based on underwater acoustic semantic communication according to claim 16, characterized in that: The text-to-image model is a Parti model, which includes extracting text features describing the text through a text encoder, and then converting the text features into an underwater image through an image generator. The text encoder is a T5 module, and the image generator includes an autoregressive Transformer module and a VQ-VAE module.

Images