Multimodal semantic communication method and device based on direct sequence spread spectrum system

Abstract

The invention discloses a multi-modal semantic communication method and device based on a direct sequence spread spectrum system, and the method comprises the steps: carrying out the semantic coding processing of multi-modal data through a semantic coding algorithm, and obtaining compressed coding data; performing spread spectrum modulation processing on the compressed coded data through direct spread spectrum system transmitting end equipment, and transmitting the modulated compressed coded data to a wireless channel; transmitting the data to the receiving end equipment of the direct spread system through the wireless channel; the direct spread system receiving end equipment carries out de-spreading demodulation processing on the received compressed coding data to obtain demodulation coding data; and performing semantic decoding processing on the demodulated and coded data through a semantic decoding algorithm to obtain the multi-modal data. According to the invention, the data volume of texts, images and videos can be effectively compressed, and multi-mode service transmission is realized in an anti-interference channel with a limited bandwidth.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent coding and decoding of satellite communication systems, and in particular to a multi-modal semantic communication method and device based on a direct sequence spread spectrum system. Background Art

[0002] Satellite communication, as an important means of mobile communication, has the advantage of wide coverage and can provide information transmission services to remote areas where base stations are not available.

[0003] At present, the traditional satellite communication system has a low transmission rate, limited service types, and low channel signal-to-noise ratio, making it difficult to support future mobile communication Internet services. In satellite communication systems, traditional text, image, and video compression coding algorithms such as Huffman coding, JPEG coding, and H256 have limited data compression and cannot achieve real-time transmission of multimodal data. In addition, traditional image source compression coding algorithms are very sensitive to signal-to-noise ratio. When the signal-to-noise ratio drops to a certain range, the image and video decoding effect will drop rapidly. Therefore, traditional source compression coding methods cannot adapt to the low bandwidth and low signal-to-noise ratio characteristics of anti-interference channels.

[0004] Therefore, how to invent a multimodal semantic communication method that can effectively compress the data volume of text, images, and videos and realize multimodal service transmission in an anti-interference channel with limited bandwidth has become an urgent problem to be solved. Summary of the invention

[0005] To this end, the present invention provides a multimodal semantic communication method and device based on a direct sequence spread spectrum system, which can effectively compress the data volume of text, images, and videos, and realize multimodal service transmission in an anti-interference channel with limited bandwidth.

[0006] In order to achieve the above object, the present invention provides the following technical solution: a multimodal semantic communication method based on a direct sequence spread spectrum system, comprising:

[0007] Perform semantic coding on the multimodal data through a semantic coding algorithm to obtain compressed coded data;

[0008] The compressed coded data is subjected to spread spectrum modulation processing by a direct spread spectrum transmission end device, and the modulated compressed coded data is sent to a wireless channel; the compressed coded data is transmitted to a direct spread spectrum reception end device through the wireless channel; the direct spread spectrum reception end device performs despreading and demodulation processing on the received compressed coded data to obtain demodulated coded data;

[0009] The demodulated coded data is semantically decoded using a semantic decoding algorithm to obtain the multimodal data.

[0010] As a preferred solution of the multimodal semantic communication method based on the direct sequence spread spectrum system, in the process of performing semantic coding processing on the multimodal data through the semantic coding algorithm, the semantic coding algorithm adopts a deep neural network model, including a text semantic coding algorithm, an image semantic coding algorithm and a video semantic coding algorithm.

[0011] As a preferred solution of the multimodal semantic communication method based on the direct sequence spread spectrum system, in the process of performing semantic decoding processing on the demodulated encoded data through the semantic decoding algorithm, the semantic decoding algorithm adopts a deep neural network model, including a text semantic decoding algorithm, an image semantic decoding algorithm and a video semantic decoding algorithm.

[0012] As a preferred solution of the multimodal semantic communication method based on the direct sequence spread spectrum system, the semantic decoding algorithm and the semantic decoding algorithm are jointly trained through the semantic knowledge base data set.

[0013] As a preferred solution of the multimodal semantic communication method based on the direct sequence spread spectrum system, in the process of semantic coding processing of the multimodal data by the semantic coding algorithm, the semantic coding processing state and result are monitored by the semantic sending end monitoring software;

[0014] In the process of performing semantic decoding processing on the demodulated coded data by using the semantic decoding algorithm, the semantic decoding processing status and results are monitored by the semantic receiving end monitoring software.

[0015] The present invention also provides a multimodal semantic communication device based on a direct sequence spread spectrum system, and the multimodal semantic communication method based on the direct sequence spread spectrum system includes:

[0016] A semantic coding module, used to perform semantic coding processing on multimodal data through a semantic coding algorithm to obtain compressed coded data;

[0017] The channel transmission module is used to perform spread spectrum modulation processing on the compressed coded data through a direct spread spectrum system transmitting end device, and send the modulated compressed coded data to a wireless channel; transmit the compressed coded data to a direct spread spectrum system receiving end device through the wireless channel; the direct spread spectrum system receiving end device performs despreading and demodulation processing on the received compressed coded data to obtain demodulated coded data;

[0018] The semantic decoding module is used to perform semantic decoding processing on the demodulated encoded data through a semantic decoding algorithm to obtain the multimodal data.

[0019] As a preferred solution for a multimodal semantic communication device based on a direct sequence spread spectrum system, in the semantic coding module, in the process of performing semantic coding processing on the multimodal data through the semantic coding algorithm, the semantic coding algorithm adopts a deep neural network model, including a text semantic coding algorithm, an image semantic coding algorithm and a video semantic coding algorithm.

[0020] As a preferred solution for a multimodal semantic communication device based on a direct sequence spread spectrum system, in the semantic decoding module, in the process of performing semantic decoding processing on the demodulated encoded data through the semantic decoding algorithm, the semantic decoding algorithm adopts a deep neural network model, including a text semantic decoding algorithm, an image semantic decoding algorithm and a video semantic decoding algorithm.

[0021] As a preferred solution for the multimodal semantic communication device based on the direct sequence spread spectrum system, in the semantic encoding module and the semantic decoding module, the semantic decoding algorithm and the semantic decoding algorithm are jointly trained through a semantic knowledge base data set.

[0022] As a preferred solution of the multimodal semantic communication device based on the direct sequence spread spectrum system, in the semantic coding module, in the process of semantic coding processing of the multimodal data by the semantic coding algorithm, the semantic coding processing state and result are monitored by the semantic sending end monitoring software;

[0023] In the semantic decoding module, during the process of performing semantic decoding processing on the demodulated coded data by using the semantic decoding algorithm, the semantic decoding processing state and result are monitored by the semantic receiving end monitoring software.

[0024] The present invention has the following advantages: semantic coding is performed on multimodal data through a semantic coding algorithm to obtain compressed coded data; the compressed coded data is spread spectrum modulated by a direct spread spectrum transmitting end device, and the modulated compressed coded data is sent to a wireless channel; the data is transmitted to a direct spread spectrum receiving end device through the wireless channel; the direct spread spectrum receiving end device despreads and demodulates the received compressed coded data to obtain demodulated coded data; the demodulated coded data is semantically decoded by a semantic decoding algorithm to obtain the multimodal data. Aiming at the problem of reliable transmission of multimodal data (text, image, video) in satellite communication, the present invention constructs a corresponding semantic coding and decoding algorithm based on a deep learning network, and realizes the compression of data bit quantity under the premise of ensuring the semantic accuracy of data transmission, so that it is more suitable for the low-bandwidth channel of the spread spectrum system. The present invention uses a direct sequence spread spectrum transmission system to ensure the reliability of data transmission, so that the communication architecture can adapt to poor channels with lower signal-to-noise ratio. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the implementation methods of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for the implementation methods or the description of the prior art. Obviously, the drawings in the following description are only exemplary, and for ordinary technicians in this field, other implementation drawings can be derived from the provided drawings without creative work.

[0026] The structures, proportions, sizes, etc. illustrated in this specification are only used to match the contents disclosed in the specification so as to facilitate understanding and reading by persons familiar with the technology. They are not used to limit the conditions under which the present invention can be implemented, and therefore have no substantial technical significance. Any structural modification, change in proportion or adjustment of size shall still fall within the scope of the technical contents disclosed in the present invention without affecting the effects and purposes that can be achieved by the present invention.

[0027] Figure 1 This is a flow chart of a multi-modal semantic communication method based on a direct sequence spread spectrum system provided in Embodiment 1 of the present invention;

[0028] Figure 2 This is a schematic diagram of a specific implementation framework of the multimodal semantic communication method based on the direct sequence spread spectrum system provided in Embodiment 1 of the present invention;

[0029] Figure 3 A schematic diagram of a text semantic coding algorithm in a multimodal semantic communication method based on a direct sequence spread spectrum system provided in Embodiment 1 of the present invention;

[0030] Figure 4 A schematic diagram of an image semantic encoding and decoding algorithm in a multimodal semantic communication method based on a direct sequence spread spectrum system provided in Embodiment 1 of the present invention;

[0031] Figure 5 A schematic diagram of a video semantic coding and decoding algorithm in a multimodal semantic communication method based on a direct sequence spread spectrum system provided in Embodiment 1 of the present invention;

[0032] Figure 6 A schematic diagram of a text semantic decoding algorithm in a multimodal semantic communication method based on a direct sequence spread spectrum system provided in Embodiment 1 of the present invention;

[0033] Figure 7 This is a schematic diagram of the architecture of a multimodal semantic communication device based on a direct sequence spread spectrum system provided in Example 2 of the present invention. DETAILED DESCRIPTION

[0034] The following is a description of the implementation of the present invention by specific embodiments. People familiar with the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0035] Example 1

[0036] See also Figure 1 and Figure 2 Embodiment 1 of the present invention provides a multimodal semantic communication method based on a direct sequence spread spectrum system, comprising the following steps:

[0037] S1. Perform semantic coding on the multimodal data through a semantic coding algorithm to obtain compressed coded data;

[0038] S2, performing spread spectrum modulation processing on the compressed coded data through a direct spread spectrum system transmitting end device, and sending the modulated compressed coded data to a wireless channel; transmitting the compressed coded data to a direct spread spectrum system receiving end device through the wireless channel; the direct spread spectrum system receiving end device performs despreading and demodulation processing on the received compressed coded data to obtain demodulated coded data;

[0039] S3. Perform semantic decoding processing on the demodulated coded data through a semantic decoding algorithm to obtain the multimodal data.

[0040] See also Figure 2 ,In this embodiment, in step S1, semantic coding is performed on the multimodal data through a semantic coding algorithm to obtain compressed coded data;

[0041] Specifically, the multimodal data is input into a semantic coding module for semantic coding processing. The semantic coding module includes a semantic coding hardware platform, a semantic coding algorithm, and a semantic sending end monitoring software.

[0042] Among them, the semantic coding hardware platform adopts the embedded ARM+GPU platform Nvidia Orin, and the semantic sending end monitoring software mainly completes the intelligent processing flow and test control, as well as the software operation status.

[0043] The semantic coding algorithm mainly includes the semantic coding processing of text, images, and videos. This part of the algorithm is deployed on the semantic coding processing hardware platform, receives the control information of the semantic sending end monitoring software to start the processing flow, and feeds back the status information and semantic coding results to the semantic sending end monitoring software.

[0044] The text semantic coding algorithm uses key technologies such as information extraction, hashing, and self-encoding to achieve reliable transmission and efficient use of large amounts of text data in low-bandwidth scenarios. Figure 3 As shown in the figure, after the sending node collects the original text of the image and text, it will go through the online operation of the three channels of "element extraction", "hash coding" and semantic compression in the text semantic coding algorithm to generate three sets of data with extremely high compression ratios: element briefing, hash coding and semantic coding, which will be transmitted to the receiving node through the direct expansion channel. The receiving node can use the above data to realize operations such as understanding the core content of the text, searching for related texts, and restoring the original text, which greatly reduces the bandwidth requirements for information transmission and improves the efficiency of text transmission, analysis and utilization.

[0045] The image semantic algorithm first obtains its feature map through feature extraction deep neural network, and then obtains the abstract semantic features of the image through semantic information extraction, including the coordinate information and text semantic description of the target in the original image. These two parts of information are cascaded to form the image encoding result, such as Figure 4 shown.

[0046] The video coding algorithm relies on deep learning technology, using target detection, action recognition, and automatic mask networks to sequentially process and extract high-level semantic features to form semantic feature tensors and text descriptions, and encodes and transmits them to wireless channels, such as Figure 5 shown.

[0047] In this embodiment, in step S2, the compressed coded data is subjected to spread spectrum modulation processing by a direct spread spectrum transmission end device, and the modulated compressed coded data is sent to a wireless channel; the compressed coded data is transmitted to a direct spread spectrum reception end device through the wireless channel; the direct spread spectrum reception end device performs despreading and demodulation processing on the received compressed coded data to obtain demodulated coded data;

[0048] Specifically, the data is compressed and encoded by the text, image, and data encoding modules and then input into the channel transmission module. After spread spectrum modulation and signal transmission, the signal is received and despread and demodulated after channel transmission, and the demodulated data is sent to the semantic decoding module.

[0049] Among them, the channel transmission module is mainly composed of three parts: the direct-sequence transmission device, the wireless channel and the direct-sequence receiving device. The direct-sequence transmission device receives the semantic information from the semantic coding module, then encodes and spread-spectrum modulates the semantic information to generate an uplink signal, and transmits it. The main function of the direct-sequence receiving device is to receive the downlink signal and perform despreading and demodulation processing, extract the semantic information from the modulated data and send it to the semantic decoding module. The wireless channel is a satellite channel with interference and noise, which can be replaced by a channel simulator in the simulation verification stage.

[0050] In this embodiment, in step S3, semantic decoding processing is performed on the demodulated coded data using a semantic decoding algorithm to obtain the multimodal data.

[0051] Specifically, after the data is transmitted through the channel, it enters the semantic decoding module at the receiving end for processing. The module includes a semantic decoding hardware platform, a semantic decoding algorithm, and a semantic receiving end monitoring software.

[0052] Among them, the hardware platform adopts a general-purpose CPU processor or an ARM processor.

[0053] The semantic decoding algorithm mainly includes the semantic decoding processing algorithm for text, image, and video. The software is deployed on the semantic decoding hardware platform, receives the control information of the semantic receiving end monitoring software to start the processing flow, and feeds back the status information and semantic extraction results to the semantic receiving end monitoring software.

[0054] The text decoding algorithm can restore the original data by simply decoding the data vector using a neural network, thereby completing the data transmission with extremely fast transmission speed. Figure 6 As shown, the received text information includes semantic coding information, hash coding information and element briefing information, among which semantic coding is mainly used for text understanding, graph construction, authenticity verification, briefing generation, etc. at the receiving end, hash coding information is used to restore the original text, and element briefing is used for text retrieval.

[0055] The image decoding algorithm uses a masked autoencoder as the basic generative model, such as Figure 4 As shown in the figure, it is a MAE generation model, which is an unsupervised learning model. It uses the input data itself as supervision to guide the neural network to try to learn a mapping relationship based on the back propagation algorithm and optimization method, so as to obtain a reconstructed output. An autoencoder is an artificial neural network with the same input and output. It learns the representation method of data through two parts of the network: encoder and decoder. The encoder compresses the input signal into a low-dimensional latent variable, and the decoder reconstructs the input signal from it. Since the autoencoder uses the training data as both input and output, it can be trained in an unsupervised manner.

[0056] The video decoding algorithm adopts an automatic mask-based training and prediction method to reconstruct and recognize video information with an extremely high masking rate. Taking into account the connection between video signals in the time and space dimensions, it uses temporal attention and spatial attention mechanisms to restore features.

[0057] In summary, the present invention performs semantic coding processing on multimodal data through a semantic coding algorithm to obtain compressed coded data; performs spread spectrum modulation processing on the compressed coded data through a direct spread spectrum system transmitting end device, and sends the modulated compressed coded data to a wireless channel; transmits to a direct spread spectrum system receiving end device through the wireless channel; the direct spread spectrum system receiving end device performs despreading and demodulation processing on the received compressed coded data to obtain demodulated coded data; performs semantic decoding processing on the demodulated coded data through a semantic decoding algorithm to obtain the multimodal data. Aiming at the problem of reliable transmission of multimodal data (text, image, video) in satellite communications, the present invention constructs a corresponding semantic coding and decoding algorithm based on a deep learning network, and realizes the compression of data bit quantity under the premise of ensuring the semantic accuracy of data transmission, so that it is more suitable for low-bandwidth channels of spread spectrum systems. The present invention uses a direct sequence spread spectrum transmission system to ensure the reliability of data transmission, so that the communication architecture can adapt to poor channels with lower signal-to-noise ratios.

[0058] It should be noted that the method of the embodiment of the present disclosure can be performed by a single device, such as a computer or a server. The method of the present embodiment can also be applied in a distributed scenario and completed by multiple devices cooperating with each other. In the case of such a distributed scenario, one of the multiple devices can only perform one or more steps in the method of the embodiment of the present disclosure, and the multiple devices will interact with each other to complete the described method.

[0059] It should be noted that the above describes some embodiments of the present disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the above embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0060] Example 2

[0061] See also Figure 7 Embodiment 2 of the present invention further provides a multi-modal semantic communication device based on a direct sequence spread spectrum system, comprising:

[0062] The semantic coding module 001 is used to perform semantic coding processing on the multimodal data through a semantic coding algorithm to obtain compressed coded data;

[0063] The channel transmission module 002 is used to perform spread spectrum modulation processing on the compressed coded data through a direct spread spectrum system transmitting end device, and send the modulated compressed coded data to a wireless channel; transmit the compressed coded data to a direct spread spectrum system receiving end device through the wireless channel; the direct spread spectrum system receiving end device performs despreading and demodulation processing on the received compressed coded data to obtain demodulated coded data;

[0064] The semantic decoding module 003 is used to perform semantic decoding processing on the demodulated encoded data through a semantic decoding algorithm to obtain the multimodal data.

[0065] In this embodiment, in the semantic coding module 001, in the process of performing semantic coding processing on the multimodal data through the semantic coding algorithm, the semantic coding algorithm adopts a deep neural network model, including a text semantic coding algorithm, an image semantic coding algorithm and a video semantic coding algorithm.

[0066] In this embodiment, in the semantic decoding module 003, in the process of performing semantic decoding processing on the demodulated encoded data through the semantic decoding algorithm, the semantic decoding algorithm adopts a deep neural network model, including a text semantic decoding algorithm, an image semantic decoding algorithm and a video semantic decoding algorithm.

[0067] In this embodiment, in the semantic encoding module 001 and the semantic decoding module 003, the semantic decoding algorithm and the semantic decoding algorithm are jointly trained through the semantic knowledge base data set.

[0068] In this embodiment, in the semantic coding module 001, in the process of semantic coding the multimodal data by the semantic coding algorithm, the semantic coding processing state and result are monitored by the semantic sending end monitoring software;

[0069] In the semantic decoding module 003, during the process of performing semantic decoding processing on the demodulated coded data by using the semantic decoding algorithm, the semantic decoding processing status and results are monitored by the semantic receiving end monitoring software.

[0070] It should be noted that the information interaction, execution process and other contents between the modules of the above-mentioned system are based on the same concept as the method embodiment in Example 1 of the present application, and the technical effects they bring are the same as those of the method embodiment of the present application. For specific contents, please refer to the description in the method embodiment shown above in the present application, and will not be repeated here.

[0071] Example 3

[0072] Embodiment 3 of the present invention provides a non-transitory computer-readable storage medium, in which a program code of a multimodal semantic communication method based on a direct sequence spread spectrum system is stored, and the program code includes instructions for executing the multimodal semantic communication method based on a direct sequence spread spectrum system of embodiment 1 or any possible implementation thereof.

[0073] The computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or a data center that includes one or more available media. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)).

[0074] Example 4

[0075] Embodiment 4 of the present invention provides an electronic device, including: a memory and a processor;

[0076] The processor and the memory communicate with each other via a bus; the memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to execute the multimodal semantic communication method based on the direct sequence spread spectrum system of embodiment 1 or any possible implementation thereof.

[0077] Specifically, the processor can be implemented by hardware or by software. When implemented by hardware, the processor can be a logic circuit, an integrated circuit, etc.; when implemented by software, the processor can be a general-purpose processor implemented by reading software codes stored in a memory. The memory can be integrated in the processor or can be located outside the processor and exist independently.

[0078] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable systems. The computer instructions can be stored in a computer-readable storage medium, or transmitted from a computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center.

[0079] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing system, they can be concentrated on a single computing system, or distributed on a network composed of multiple computing systems, and optionally, they can be implemented by a program code executable by a computing system, so that they can be stored in a storage system and executed by the computing system, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0080] Although the present invention has been described in detail above by general description and specific embodiments, it is obvious to those skilled in the art that some modifications or improvements can be made to the present invention. Therefore, these modifications or improvements made without departing from the spirit of the present invention all belong to the scope of protection claimed by the present invention.

Claims

1. A multimodal semantic communication method based on a direct sequence spread spectrum system, characterized in that: include: Perform semantic coding on the multimodal data through a semantic coding algorithm to obtain compressed coded data; Performing spread spectrum modulation processing on the compressed coded data by a direct spread spectrum system transmitting end device, and sending the modulated compressed coded data to a wireless channel; The received compressed coded data are transmitted to a receiving end device of a direct spread spectrum system through the wireless channel; the receiving end device of the direct spread spectrum system performs despreading and demodulation processing on the received compressed coded data to obtain demodulated coded data; The demodulated coded data is semantically decoded using a semantic decoding algorithm to obtain the multimodal data.

2. The multimodal semantic communication method based on direct sequence spread spectrum system according to claim 1, characterized in that: In the process of performing semantic coding processing on the multimodal data through the semantic coding algorithm, the semantic coding algorithm adopts a deep neural network model, including a text semantic coding algorithm, an image semantic coding algorithm and a video semantic coding algorithm.

3. The multimodal semantic communication method based on direct sequence spread spectrum system according to claim 2 is characterized in that: In the process of performing semantic decoding processing on the demodulated encoded data through the semantic decoding algorithm, the semantic decoding algorithm adopts a deep neural network model, including a text semantic decoding algorithm, an image semantic decoding algorithm and a video semantic decoding algorithm.

4. The multimodal semantic communication method based on direct sequence spread spectrum system according to claim 3 is characterized in that: The semantic decoding algorithm and the semantic decoding algorithm are jointly trained using a semantic knowledge base dataset.

5. The multimodal semantic communication method based on direct sequence spread spectrum system according to claim 4 is characterized in that: In the process of semantic coding the multimodal data by the semantic coding algorithm, the semantic coding processing state and result are monitored by the semantic sending end monitoring software; In the process of performing semantic decoding processing on the demodulated coded data by using the semantic decoding algorithm, the semantic decoding processing status and results are monitored by the semantic receiving end monitoring software.

6. A multimodal semantic communication device based on a direct sequence spread spectrum system, adopting a multimodal semantic communication method based on a direct sequence spread spectrum system as claimed in any one of claims 1 to 5, characterized in that: include: A semantic coding module, used to perform semantic coding processing on multimodal data through a semantic coding algorithm to obtain compressed coded data; A channel transmission module, used for performing spread spectrum modulation processing on the compressed coded data through a direct spread spectrum system transmitting end device, and sending the modulated compressed coded data to a wireless channel; The received compressed coded data are transmitted to a receiving end device of a direct spread spectrum system through the wireless channel; the receiving end device of the direct spread spectrum system performs despreading and demodulation processing on the received compressed coded data to obtain demodulated coded data; The semantic decoding module is used to perform semantic decoding processing on the demodulated encoded data through a semantic decoding algorithm to obtain the multimodal data.

7. The multimodal semantic communication device based on direct sequence spread spectrum system according to claim 6, characterized in that: In the semantic coding module, in the process of performing semantic coding processing on the multimodal data through the semantic coding algorithm, the semantic coding algorithm adopts a deep neural network model, including a text semantic coding algorithm, an image semantic coding algorithm and a video semantic coding algorithm.

8. The multimodal semantic communication device based on direct sequence spread spectrum system according to claim 7, characterized in that: In the semantic decoding module, in the process of performing semantic decoding processing on the demodulated encoded data through the semantic decoding algorithm, the semantic decoding algorithm adopts a deep neural network model, including a text semantic decoding algorithm, an image semantic decoding algorithm and a video semantic decoding algorithm.

9. The multimodal semantic communication device based on direct sequence spread spectrum system according to claim 8, characterized in that: In the semantic encoding module and the semantic decoding module, the semantic decoding algorithm and the semantic decoding algorithm are jointly trained through a semantic knowledge base data set.

10. The multimodal semantic communication device based on direct sequence spread spectrum system according to claim 9, characterized in that: In the semantic coding module, in the process of semantic coding the multimodal data by the semantic coding algorithm, the semantic coding processing state and result are monitored by the semantic sending end monitoring software; In the semantic decoding module, during the process of performing semantic decoding processing on the demodulated coded data by using the semantic decoding algorithm, the semantic decoding processing state and result are monitored by the semantic receiving end monitoring software.

Images