Semantic communication method and apparatus, electronic device, and storage medium

Abstract

The application provides a semantic communication method and device, electronic equipment and storage medium. The method comprises the following steps: obtaining encrypted data based on a received source signal; performing semantic encoding on the encrypted data to obtain an encrypted semantic vector; performing channel encoding on the encrypted semantic vector to obtain encrypted semantic information; and sending the encrypted semantic information to a receiving end. The method further comprises the following steps: receiving the encrypted semantic information sent by the sending end; performing channel decoding on the encrypted semantic information to obtain first secret semantic information; performing semantic decoding on the first secret semantic information by using a coding and decoding network structure constructed based on a public knowledge base to obtain second secret semantic information; decrypting the second secret semantic information to obtain plaintext semantic information; and performing local semantic decoding on the plaintext semantic information to obtain semantic private decoding information. The semantic communication method provided by the application reduces the potential risk of data privacy leakage when the semantic communication is coded and decoded on a third-party platform.

Description

Technical Field

[0001] This invention relates to the field of communication technology, and in particular to a semantic communication method, apparatus, electronic device, and storage medium. Background Technology

[0002] In existing semantic communication frameworks, semantic extraction, semantic encoding, and channel encoding are performed at the sending end. After transmission through the physical channel, the signal reaches the receiving end, where channel decoding, semantic decoding, and semantic reconstruction are performed. Current work on semantic communication focuses on semantic expression, semantic feature extraction and reconstruction, and semantic source-channel encoding. However, there is still limited research on the privacy leakage issues involved in semantic transmission. Nevertheless, privacy protection during semantic transmission remains one of the key issues in semantic communication.

[0003] Compared to traditional communication systems, semantic end-to-end communication systems utilize joint transceivers constructed using deep learning to replace traditional physical layer independent communication modules. However, semantic encoding and decoding, as well as knowledge base updates, consume significant computing and storage resources, often requiring operation on third-party platforms such as edge servers. This process raises concerns about exposing user privacy data to untrusted third parties. Furthermore, to better encode and decode semantic information, third parties may need to infer the implicit semantics of messages from their private knowledge bases. Summary of the Invention

[0004] This invention provides a semantic communication method, apparatus, electronic device, and storage medium to solve the technical problem of privacy data leakage risk during semantic transmission in the prior art.

[0005] This invention provides a semantic communication method applied at the sending end, comprising:

[0006] Encrypted data is obtained based on the received source signal;

[0007] An encoding / decoding network structure based on a public knowledge base is used to semantically encode the encrypted data to obtain an encrypted semantic vector;

[0008] Channel coding is performed on the encrypted semantic vector to obtain encrypted semantic information;

[0009] The encrypted semantic information is sent to the receiving end.

[0010] In some embodiments, obtaining encrypted data based on the received source signal includes:

[0011] When the amount of data in the source signal exceeds the target threshold, semantic extraction is performed on the source signal to obtain the hidden layer semantics output by the intermediate layer of the encoding / decoding network structure.

[0012] The hidden semantics are encrypted to obtain the encrypted data.

[0013] In some embodiments, obtaining encrypted data based on the received source signal includes:

[0014] If the amount of data in the source signal does not exceed the target threshold, the source signal is encrypted to obtain the encrypted data.

[0015] In some embodiments, the encryption method of the source signal includes any one of the following:

[0016] Differential privacy processing, data anonymization processing, homomorphic encryption, and secure multi-party computation.

[0017] In some embodiments, when the encryption method is homomorphic encryption, the codec network structure is constructed in the following manner:

[0018] Set the activation function of the encoding / decoding network structure to a homomorphic operation function;

[0019] Replace the max pooling layer in the encoding / decoding network structure with an average pooling layer.

[0020] This invention provides a semantic communication method applied at a receiving end, comprising:

[0021] Receive encrypted semantic information sent by the sender;

[0022] Channel decoding is performed on the encrypted semantic information to obtain the first encrypted semantic information;

[0023] A codec network structure based on a public knowledge base is used to semantically decode the first secret semantic information to obtain the second secret semantic information;

[0024] The second encrypted semantic information is decrypted to obtain the plaintext semantic information;

[0025] A decoding network structure based on a private knowledge base is used to perform local semantic decoding on the plaintext semantic information to obtain semantic private decoding information.

[0026] The present invention also provides a semantic communication device applied at a sending end, comprising:

[0027] The encryption module is used to obtain encrypted data based on the received source signal;

[0028] The first encoding module is used to perform semantic encoding on the encrypted data using an encoding / decoding network structure built on a public knowledge base, to obtain an encrypted semantic vector;

[0029] The second encoding module is used to perform channel coding on the encrypted semantic vector to obtain encrypted semantic information;

[0030] The sending module is used to send the encrypted semantic information to the receiving end.

[0031] The present invention also provides a semantic communication device applied at a receiving end, comprising:

[0032] The receiving module is used to receive encrypted semantic information sent by the sending end;

[0033] The first decoding module is used to perform channel decoding on the encrypted semantic information to obtain the first encrypted semantic information;

[0034] The second decoding module is used to perform semantic decoding on the first secret semantic information using an encoding and decoding network structure built on a public knowledge base, so as to obtain the second secret semantic information.

[0035] The decryption module is used to decrypt the second encrypted semantic information to obtain plaintext semantic information;

[0036] The third decoding module is used to perform local semantic decoding on the plaintext semantic information using a decoding network structure built on a private knowledge base, so as to obtain semantic private decoding information.

[0037] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the semantic communication method as described above.

[0038] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the semantic communication method as described above.

[0039] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the semantic communication method as described above.

[0040] The semantic communication method, apparatus, electronic device, and storage medium provided by this invention encode and decode encrypted data, rather than directly encode and decode the semantics of the original data. Furthermore, at the receiving end, instead of directly combining with a public knowledge base to recover semantic information, semantic information with local private interpretation is recovered by combining with the receiving end's private knowledge base for semantic reasoning. This reduces the potential risk of data privacy leakage when semantic communication is encoded and decoded on a third-party platform, and enables local private decoding of the received semantic information. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0042] Figure 1 This is an interactive schematic diagram of the semantic communication method provided by the present invention;

[0043] Figure 2 This is one of the flowcharts illustrating the semantic communication method provided by the present invention;

[0044] Figure 3 This is one of the schematic diagrams of a communication framework applying the semantic communication method provided by the present invention;

[0045] Figure 4 This is the second schematic diagram of the communication framework for applying the semantic communication method provided by this invention;

[0046] Figure 5 This is the second flowchart illustrating the semantic communication method provided by the present invention;

[0047] Figure 6 This is one of the structural schematic diagrams of the semantic communication device provided by the present invention;

[0048] Figure 7 This is the second schematic diagram of the semantic communication device provided by the present invention;

[0049] Figure 8 This is a schematic diagram of the physical structure of the electronic device provided by the present invention. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0051] Figure 1 This is a schematic diagram illustrating the interaction of the semantic communication method provided by the present invention. (Refer to...) Figure 1 The overall process of the semantic communication method provided by this invention is as follows:

[0052] The transmitting end encrypts the received source signal, performs semantic encoding on the encrypted information using an encoder in conjunction with a public knowledge base to obtain an encrypted semantic vector, and then performs channel encoding on the encrypted semantic vector.

[0053] The encrypted semantic information obtained after channel coding is transmitted to the receiving end through the physical transmission channel;

[0054] The receiver's decoder performs channel decoding on the received encrypted semantic information to obtain the first encrypted semantic information; then, it performs semantic decoding in conjunction with a public knowledge base to obtain the second encrypted semantic information; then, it decrypts the second encrypted semantic information to obtain the plaintext semantic information; and finally, it performs local semantic decoding on the plaintext semantic information in conjunction with a private knowledge base to obtain the semantically private decoded information.

[0055] Figure 2 This is one of the flowcharts illustrating the semantic communication method provided by the present invention. (Refer to...) Figure 2 The semantic communication method provided by the present invention is applied to the sending end, and the method may include steps 210, 220, 230 and 240.

[0056] Step 210: Obtain encrypted data based on the received source signal;

[0057] Step 220: Using an encoding / decoding network structure built on a public knowledge base, semantically encode the encrypted data to obtain an encrypted semantic vector;

[0058] Step 230: Channel-encode the encrypted semantic vector to obtain encrypted semantic information;

[0059] Step 240: Send the encrypted semantic information to the receiving end.

[0060] It should be noted that the subject executing the semantic communication method provided by this invention can be an electronic device, a component within an electronic device, an integrated circuit, or a chip. The electronic device can be a mobile electronic device or a non-mobile electronic device. For example, a mobile electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc., while a non-mobile electronic device can be a server, network attached storage (NAS), or personal computer (PC), etc. This invention does not impose specific limitations.

[0061] The technical solution of this invention will be described in detail below using the example of a computer executing the semantic communication method provided by this invention.

[0062] like Figure 3As shown, the semantic communication method provided by this invention can be implemented based on a trained semantic communication framework.

[0063] This communication framework consists of two parts: a transmitter and a receiver. The transmitter may include a transmitting terminal and an encoder, while the receiver may include a receiving terminal and a decoder. The transmitter and receiver transmit data through a physical transmission channel.

[0064] In step 210, the transmitting terminal encrypts the received source signal to obtain the encrypted source signal, i.e., encrypted data.

[0065] The source signal may include, but is not limited to, text, images, audio, video, or combinations of the above types of signals.

[0066] In practice, data can be encrypted using random perturbation or cryptographic encryption.

[0067] Data perturbation encryption can protect data privacy by modifying the original information by adding random noise, but the perturbed data still retains the distribution information of the original data.

[0068] Data encrypted using cryptography can be computed on encrypted data without accessing the original data.

[0069] In step 220, the transmitting terminal transmits encrypted data to the encoder, which can be used for semantic coding and channel coding respectively.

[0070] The encoder combines a public knowledge base in the server with an encoding / decoding network structure built on the public knowledge base to perform semantic encoding on the encrypted data of the ciphertext state, resulting in an encrypted semantic vector.

[0071] An encoder is a device that encodes signals or data and converts them into a signal form that can be used for communication, transmission, and storage.

[0072] The encoding / decoding network structure built on a public knowledge base can include the encoding network structure corresponding to the sending end and the decoding network structure corresponding to the receiving end.

[0073] In step 230, the encoder performs channel coding on the encrypted semantic vector to obtain encrypted semantic information.

[0074] In step 240, the encoder transmits the encrypted semantic information to the decoder at the receiving end via the physical transmission channel.

[0075] After the decoder receives the encrypted semantic information sent by the sender, it performs channel decoding and semantic decoding on the encrypted semantic information.

[0076] In practice, the decoder first performs channel decoding on the received encrypted semantic information to obtain the first encrypted semantic information. The decoder then combines the public knowledge base in the server and the encoding / decoding network structure built on the public knowledge base to perform semantic decoding on the first encrypted semantic information to recover the received primary encrypted semantic information, i.e., the second encrypted semantic information.

[0077] The second encrypted semantic information is transmitted to the receiving terminal. After the receiving terminal decrypts the second encrypted semantic information, it obtains the plaintext semantic information.

[0078] The receiving terminal then combines its own private knowledge base to perform local semantic decoding on the received plaintext semantic information, and then performs semantic reconstruction to finally obtain the semantic private decoded information.

[0079] Based on the traditional semantic communication framework, this invention adds encryption to the source signal at the sending end, decryption to the encrypted semantics at the receiving end, and local semantic interpretation combined with the receiving end's private knowledge base.

[0080] The semantic communication method provided by this invention, by encoding and decoding encrypted data, no longer directly encodes and decodes the semantics of the original data. Moreover, at the receiving end, instead of directly recovering semantic information by combining it with a public knowledge base, it recovers semantic information with local private interpretation by combining it with the receiving end's private knowledge base for semantic reasoning. This reduces the potential risk of data privacy leakage when semantic communication is encoded and decoded on a third-party platform, and enables local private decoding of the received semantic information.

[0081] In some embodiments, encrypted data is obtained based on the received source signal, including:

[0082] If the amount of data in the source signal does not exceed the target threshold, the source signal is encrypted to obtain encrypted data.

[0083] It should be noted that the target threshold can be set according to user needs. After setting the target threshold, the current communication scenario can be determined to be a small amount of data transmission if the amount of data in the source signal does not exceed the target threshold.

[0084] In practical implementation, the semantic communication method provided by the embodiments of the present invention can be applied to the following communication scenarios: dense semantic communication scenarios with a small amount of data transmission;

[0085] In scenarios involving small amounts of data transmission, the bandwidth consumed by encrypted data transmission between the sending terminal and the server can be disregarded, and can be based on... Figure 2 The communication framework shown encrypts the source signal.

[0086] In some embodiments, the encryption method for the source signal can be random perturbation encryption or cryptographic encryption.

[0087] Random perturbation encryption of data can include differential privacy techniques or data anonymization techniques. When this type of encrypted data is manipulated in a semantic communication codec, the perturbation mechanism reduces data usability and degrades the training accuracy of the codec. Therefore, it is necessary to limit the noise threshold and train a semantic codec that satisfies this type of perturbation encryption.

[0088] Differential privacy is a technique in cryptography that aims to maximize the accuracy of data queries when querying statistical databases, while minimizing the chance of identifying the records.

[0089] Data anonymization is the process of protecting private or sensitive information by eliminating or encrypting the identifiers that link individuals to stored data.

[0090] Cryptographic encryption of data can include homomorphic encryption or secure multi-party computation. Operating on such encrypted data in a semantic communication codec involves nonlinear computations beyond the computational capabilities of the encrypted data. Therefore, during training, linear operations are needed to approximate the relevant functions, ensuring that the computational results in the encrypted space can be correctly decoded.

[0091] Homomorphic encryption refers to homomorphically encrypting the original data, performing specific operations on the ciphertext, and then homomorphically decrypting the resulting ciphertext to obtain plaintext that is equivalent to the data result obtained by directly performing the same calculation on the original plaintext data.

[0092] Secure multi-party computation primarily addresses the problem of how to securely compute an agreed-upon function without a trusted third party. Secure multi-party computation can simultaneously ensure input privacy and computational accuracy. Using mathematical theory, it guarantees that the input information of each participating party is not exposed, while simultaneously obtaining accurate computation results, even without a trusted third party.

[0093] Understandably, different encryption methods will also require adaptive adjustments or changes to the encoding and decoding network structure.

[0094] It should be noted that the encryption methods for source signals provided in this implementation include, but are not limited to, differential privacy processing, data anonymization processing, homomorphic encryption, and secure multi-party computation. Other encryption methods may also be used, which can be determined according to actual needs and are not specifically limited here.

[0095] The semantic communication method provided by this invention effectively protects the source data by encrypting it, thereby reducing the potential risk of data leakage.

[0096] In some embodiments, encrypted data is obtained based on the received source signal, including:

[0097] When the amount of data in the source signal exceeds the target threshold, semantic extraction is performed on the source signal to obtain the hidden layer semantics output by the intermediate layer of the encoding / decoding network structure.

[0098] The hidden semantics are encrypted to obtain encrypted data.

[0099] In practice, the semantic communication method provided by the embodiments of the present invention can be applied to dense semantic communication scenarios involving large amounts of data transmission.

[0100] It should be noted that the target threshold can be set according to user needs. After setting the target threshold, the current communication scenario can be determined to be a large amount of data transmission when the amount of data in the source signal exceeds the target threshold.

[0101] In scenarios involving large-scale data transmission, the communication overhead of transmitting encrypted data between the terminal and the server is significant. Semantic communication frameworks for this scenario, such as... Figure 4 As shown.

[0102] Understandably, based on Figure 4 The semantic communication framework shown can use the same encryption method as the above embodiments to encrypt the source signal, and will not be described again here.

[0103] In some embodiments, for Figure 4 The semantic communication framework shown is similar to... Figure 3 The semantic communication framework shown differs from the one shown in that it first extracts the semantics of the source signal to obtain the hidden semantics of the intermediate layer output of the encoding and decoding network structure;

[0104] The hidden semantics are then encrypted to obtain encrypted data.

[0105] Within this communication framework, the semantic communication method provided by this invention may include the following steps:

[0106] Step 1: The transmitting terminal performs preliminary semantic extraction on the received source signal based on split learning to obtain the hidden layer semantics output by the intermediate layer of the encoder-decoder network structure. The hidden layer semantics calculated by the first few layers of the neural network are then encrypted using homomorphic encryption or other methods to obtain encrypted data.

[0107] The intermediate layer can be at least one layer of a neural network in the encoding / decoding network structure, excluding the input layer and the output layer.

[0108] Step 2: The sending terminal transmits the encrypted data to the encoder. The encoder, in conjunction with a public knowledge base, performs semantic encoding on the encrypted data using several layers of a neural network to obtain an encrypted semantic vector. The encoder then performs channel encoding on the encrypted semantic vector to obtain the encrypted semantic information.

[0109] Step 3: The encoder transmits the encrypted semantic information to the decoder through the physical transmission channel;

[0110] Step 4: The decoder performs channel decoding on the received encrypted semantic information, and then performs semantic decoding in conjunction with the public knowledge base to recover the received primary encrypted semantic information;

[0111] Step 5: Transmit the primary encrypted semantic information to the receiving terminal. After the receiving terminal decrypts the primary encrypted semantic information, it obtains the plaintext semantic information.

[0112] By combining its own private knowledge base, the system performs local semantic decoding on the received plaintext semantic information, and then performs semantic reconstruction to finally obtain the semantic private decoded information.

[0113] The semantic communication method provided by this invention can employ different communication frameworks and semantic communication methods in scenarios involving encrypted transmission of different data sources, thereby improving the applicability of semantic communication.

[0114] In some embodiments, when the encryption method is homomorphic encryption, the codec network structure is constructed as follows:

[0115] Set the activation function of the encoding / decoding network structure to a homomorphic operation function;

[0116] Replace the max pooling layer in the encoding / decoding network structure with an average pooling layer.

[0117] The training process of the encoding / decoding network structure provided by this invention is illustrated by taking homomorphic encryption of the source signal as an example.

[0118] Training Step 1: Select a training dataset as a public knowledge base; the source signal types in the training dataset can be determined according to different semantic communication scenarios.

[0119] Training Step 2: Construct an encoding / decoding network structure based on a public knowledge base, and a decoding network structure based on a private knowledge base;

[0120] In practice, the activation functions in the codec network structure are replaced with homomorphic operation functions. Homomorphic operation functions can be approximate functions composed of homomorphic addition and homomorphic multiplication, and the max pooling layers in the codec network structure are replaced with average pooling layers.

[0121] Initialize the model parameters of the encoder-decoder network structure;

[0122] Training step 3: Encrypt the source signal using homomorphic encryption to obtain encrypted data;

[0123] Training step 4: Use the encoder to process the encrypted data to obtain the encrypted semantic vector;

[0124] Training step 5: Use the decoder to process the encrypted semantic vector to obtain the initial encrypted semantic information;

[0125] Training step 6: Use homomorphic decryption to decrypt the initial encrypted semantic information to obtain the plaintext semantic information;

[0126] Training Step 7: Use a decoding network structure based on a private knowledge base to perform local semantic decoding on plaintext semantic information to obtain semantic private decoding information;

[0127] Training Step 8: Repeat training steps 1-7 until the model parameters of the encoder-decoder network structure based on the public knowledge base and the decoder network structure based on the private knowledge base converge or the maximum number of training iterations is reached.

[0128] It is understandable that the activation function and max pooling layer in existing codec networks correspond to nonlinear operations. After homomorphically encrypting the source signal, in order to adapt to the linear operations that homomorphic encryption can only perform, the activation function is replaced with an approximate polynomial function, and the max pooling layer is replaced with an average pooling layer. This can play a role in protecting the privacy of the input source signal from being leaked to third parties.

[0129] This method can significantly reduce the risk of privacy leaks of user-transmitted data while only increasing the number of parameters and server processing time by a small amount.

[0130] In the encryption methods provided in the above embodiments, the training process of the encoding and decoding network structure provided by the present invention is illustrated by taking differential privacy processing of the source signal as an example.

[0131] Training Step 1: Select a training dataset as a public knowledge base; the source signal types in the training dataset can be determined according to different semantic communication scenarios.

[0132] Training Step 2: Construct an encoding / decoding network structure based on a public knowledge base and a decoding network structure based on a private knowledge base, and initialize the parameters of the encoding / decoding network structure;

[0133] Training Step 3: Use differential privacy technology to process the source signals in the training dataset, add noise according to the sensitivity of the dataset, and obtain the perturbed encrypted data;

[0134] Training step 4: Use the encoder to process the encrypted data to obtain the encrypted semantic vector;

[0135] Training step 5: Use the decoder to process the encrypted semantic vector to obtain the initial encrypted semantic information;

[0136] Training step 6: Use a decoding network structure based on a private knowledge base to perform local semantic decoding on plaintext semantic information, and finally obtain semantic private decoding information;

[0137] Training Step 7: Repeat training steps 1-6 until the model parameters of the encoder-decoder network structure based on the public knowledge base and the decoder network structure based on the private knowledge base converge or the maximum number of training iterations is reached.

[0138] This invention proposes a privacy-preserving communication architecture for encrypted semantics based on semantic communication. It integrates data encryption / decryption with a local semantic decoding module incorporating a private knowledge base into a traditional semantic communication framework. Based on this framework, privacy protection is achieved for data semantically encoded and decoded on third-party platforms. Simultaneously, local semantic decoding of received information is achieved without exposing the receiver's private knowledge base to third parties. Compared to the original semantic communication architecture, the privacy of user data and the private knowledge base is effectively protected, and this framework has broad applicability in scenarios involving encrypted transmission of varying data volumes.

[0139] Figure 5 This is the second flowchart illustrating the semantic communication method provided by the present invention. (Refer to...) Figure 5 The semantic communication method provided by the present invention is applied to the receiving end, and the method may include: step 510, step 520, step 530, step 540 and step 550.

[0140] Step 510: Receive encrypted semantic information sent by the sending end;

[0141] Step 520: Perform channel decoding on the encrypted semantic information to obtain the first encrypted semantic information;

[0142] Step 530: Using an encoding / decoding network structure built on a public knowledge base, semantically decode the first secret semantic information to obtain the second secret semantic information;

[0143] Step 540: Decrypt the second secret state semantic information to obtain the plaintext semantic information;

[0144] Step 550: Using a decoding network structure built on a private knowledge base, perform local semantic decoding on plaintext semantic information to obtain semantic private decoding information.

[0145] It should be noted that the subject executing the semantic communication method provided by this invention can be an electronic device, a component within an electronic device, an integrated circuit, or a chip. The electronic device can be a mobile electronic device or a non-mobile electronic device. For example, a mobile electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc., while a non-mobile electronic device can be a server, network attached storage (NAS), or personal computer (PC), etc. This invention does not impose specific limitations.

[0146] The technical solution of this invention will be described in detail below using the example of a computer executing the semantic communication method provided by this invention.

[0147] like Figure 3 As shown, the semantic communication method provided by this invention can be implemented based on a trained semantic communication framework.

[0148] This communication framework consists of two parts: a transmitter and a receiver. The transmitter may include a transmitting terminal and an encoder, while the receiver may include a receiving terminal and a decoder. The transmitter and receiver transmit data through a physical transmission channel.

[0149] In this embodiment, the steps before the receiving end receives the encrypted semantic information sent by the sending end are the same as in the above embodiment, and will not be repeated here.

[0150] In step 510, the decoder at the receiving end receives the encrypted semantic information sent by the sending end.

[0151] In step 520, the decoder performs channel decoding on the encrypted semantic information to obtain the first encrypted semantic information.

[0152] In step 530, the decoder then uses the encoding / decoding network structure built from a public knowledge base to perform semantic decoding on the first encrypted semantic information to obtain the second encrypted semantic information. The decoder then sends the second encrypted semantic information to the receiving terminal.

[0153] In step 540, the receiving terminal decrypts the second encrypted semantic information to obtain the plaintext semantic information. The decryption method can be determined based on the encryption method, and is not specifically limited here.

[0154] In step 550, the receiving terminal uses a decoding network structure built on a private knowledge base to perform local semantic decoding on the plaintext semantic information, and then performs semantic reconstruction to finally obtain the semantic private decoded information.

[0155] The semantic communication method provided by this invention, by encoding and decoding encrypted data, no longer directly encodes and decodes the semantics of the original data. Moreover, at the receiving end, instead of directly recovering semantic information by combining it with a public knowledge base, it recovers semantic information with local private interpretation by combining it with the receiving end's private knowledge base for semantic reasoning. This reduces the potential risk of data privacy leakage when semantic communication is encoded and decoded on a third-party platform, and enables local private decoding of the received semantic information.

[0156] The semantic communication device provided by the present invention is described below. The semantic communication device described below and the semantic communication method described above can be referred to in correspondence.

[0157] Figure 6 This is one of the structural schematic diagrams of the semantic communication device provided by the present invention. (Refer to...) Figure 6 The semantic communication device provided by the present invention is applied to the sending end. The device may include: an encryption module 610, a first encoding module 620, a second encoding module 630, and a sending module 640.

[0158] The encryption module 610 is used to obtain encrypted data based on the received source signal;

[0159] The first encoding module 620 is used to perform semantic encoding on the encrypted data using an encoding / decoding network structure built on a public knowledge base to obtain an encrypted semantic vector;

[0160] The second encoding module 630 is used to perform channel coding on the encrypted semantic vector to obtain encrypted semantic information;

[0161] The sending module 640 is used to send the encrypted semantic information to the receiving end.

[0162] The semantic communication device provided by this invention encodes and decodes encrypted data, rather than directly encoding and decoding the semantics of the original data. Furthermore, at the receiving end, instead of directly combining with a public knowledge base to recover semantic information, it performs semantic reasoning based on the receiving end's private knowledge base to recover semantic information with local private interpretation. This reduces the potential risk of data privacy leakage when semantic communication is encoded and decoded on a third-party platform, and enables local private decoding of the received semantic information.

[0163] In some embodiments, the encryption module 610 is further configured to:

[0164] When the amount of data in the source signal exceeds the target threshold, semantic extraction is performed on the source signal to obtain the hidden layer semantics output by the intermediate layer of the encoding / decoding network structure.

[0165] The hidden semantics are encrypted to obtain the encrypted data.

[0166] In some embodiments, the encryption module 610 is further configured to:

[0167] If the amount of data in the source signal does not exceed the target threshold, the source signal is encrypted to obtain the encrypted data.

[0168] In some embodiments, the encryption method of the source signal includes any one of the following:

[0169] Differential privacy processing, data anonymization processing, homomorphic encryption, and secure multi-party computation.

[0170] In some embodiments, when the encryption method is homomorphic encryption, the codec network structure is constructed in the following manner:

[0171] Set the activation function of the encoding / decoding network structure to a homomorphic operation function;

[0172] Replace the max pooling layer in the encoding / decoding network structure with an average pooling layer.

[0173] Figure 7 This is a second schematic diagram of the semantic communication device provided by the present invention. (Refer to...) Figure 7 The semantic communication device provided by the present invention is applied to a receiving end. The method may include: a receiving module 710, a first decoding module 720, a second decoding module 730, a decryption module 740, and a third decoding module 750.

[0174] The receiving module 710 is used to receive encrypted semantic information sent by the sending end;

[0175] The first decoding module 720 is used to perform channel decoding on the encrypted semantic information to obtain the first encrypted semantic information;

[0176] The second decoding module 730 is used to perform semantic decoding on the first secret semantic information using an encoding and decoding network structure built on a public knowledge base to obtain the second secret semantic information.

[0177] The decryption module 740 is used to decrypt the second encrypted semantic information to obtain plaintext semantic information;

[0178] The third decoding module 750 is used to perform local semantic decoding on the plaintext semantic information using a decoding network structure built on a private knowledge base, so as to obtain semantic private decoding information.

[0179] The semantic communication device provided by this invention encodes and decodes encrypted data, rather than directly encoding and decoding the semantics of the original data. Furthermore, at the receiving end, instead of directly combining with a public knowledge base to recover semantic information, it performs semantic reasoning based on the receiving end's private knowledge base to recover semantic information with local private interpretation. This reduces the potential risk of data privacy leakage when semantic communication is encoded and decoded on a third-party platform, and enables local private decoding of the received semantic information.

[0180] Figure 8 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 8 As shown, the electronic device may include: a processor 810, a communications interface 820, a memory 830, and a communication bus 840, wherein the processor 810, the communications interface 820, and the memory 830 communicate with each other via the communication bus 840. The processor 810 can call logical instructions in the memory 830 to execute a semantic communication method, which includes:

[0181] Encrypted data is obtained based on the received source signal;

[0182] An encoding / decoding network structure based on a public knowledge base is used to semantically encode the encrypted data to obtain an encrypted semantic vector;

[0183] Channel coding is performed on the encrypted semantic vector to obtain encrypted semantic information;

[0184] The encrypted semantic information is sent to the receiving end.

[0185] Alternatively, the method may include:

[0186] Receive encrypted semantic information sent by the sender;

[0187] Channel decoding is performed on the encrypted semantic information to obtain the first encrypted semantic information;

[0188] A codec network structure based on a public knowledge base is used to semantically decode the first secret semantic information to obtain the second secret semantic information;

[0189] The second encrypted semantic information is decrypted to obtain the plaintext semantic information;

[0190] A decoding network structure based on a private knowledge base is used to perform local semantic decoding on the plaintext semantic information to obtain semantic private decoding information.

[0191] Furthermore, the logical instructions in the aforementioned memory 830 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0192] On the other hand, the present invention also provides a computer program product, the computer program product comprising a computer program that can be stored on a non-transitory computer-readable storage medium, wherein when the computer program is executed by a processor, the computer is able to execute the semantic communication method provided by the above methods, the method comprising:

[0193] Encrypted data is obtained based on the received source signal;

[0194] An encoding / decoding network structure based on a public knowledge base is used to semantically encode the encrypted data to obtain an encrypted semantic vector;

[0195] Channel coding is performed on the encrypted semantic vector to obtain encrypted semantic information;

[0196] The encrypted semantic information is sent to the receiving end.

[0197] Alternatively, the method may include:

[0198] Receive encrypted semantic information sent by the sender;

[0199] Channel decoding is performed on the encrypted semantic information to obtain the first encrypted semantic information;

[0200] A codec network structure based on a public knowledge base is used to semantically decode the first secret semantic information to obtain the second secret semantic information;

[0201] The second encrypted semantic information is decrypted to obtain the plaintext semantic information;

[0202] A decoding network structure based on a private knowledge base is used to perform local semantic decoding on the plaintext semantic information to obtain semantic private decoding information.

[0203] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the semantic communication methods provided by the methods described above, the method comprising:

[0204] Encrypted data is obtained based on the received source signal;

[0205] An encoding / decoding network structure based on a public knowledge base is used to semantically encode the encrypted data to obtain an encrypted semantic vector;

[0206] Channel coding is performed on the encrypted semantic vector to obtain encrypted semantic information;

[0207] The encrypted semantic information is sent to the receiving end.

[0208] Alternatively, the method may include:

[0209] Receive encrypted semantic information sent by the sender;

[0210] Channel decoding is performed on the encrypted semantic information to obtain the first encrypted semantic information;

[0211] A codec network structure based on a public knowledge base is used to semantically decode the first secret semantic information to obtain the second secret semantic information;

[0212] The second encrypted semantic information is decrypted to obtain the plaintext semantic information;

[0213] A decoding network structure based on a private knowledge base is used to perform local semantic decoding on the plaintext semantic information to obtain semantic private decoding information.

[0214] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0215] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0216] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A semantic communication method, characterized in that, Applied to the sending end, including: Encrypted data is obtained based on the received source signal; An encoding / decoding network structure based on a public knowledge base is used to semantically encode the encrypted data to obtain an encrypted semantic vector; Channel coding is performed on the encrypted semantic vector to obtain encrypted semantic information; Send the encrypted semantic information to the receiving end; The process of obtaining encrypted data based on the received source signal includes: When the amount of data in the source signal exceeds the target threshold, semantic extraction is performed on the source signal to obtain the hidden layer semantics output by the intermediate layer of the encoding / decoding network structure; the hidden layer semantics are then encrypted to obtain the encrypted data. When the encryption method is homomorphic encryption, the codec network structure is constructed as follows: the activation function of the codec network structure is set to a homomorphic operation function; the max pooling layer in the codec network structure is replaced with an average pooling layer.

2. The semantic communication method according to claim 1, characterized in that, The process of obtaining encrypted data based on the received source signal includes: If the amount of data in the source signal does not exceed the target threshold, the source signal is encrypted to obtain the encrypted data.

3. The semantic communication method according to claim 1 or 2, characterized in that, The encryption method of the source signal includes any one of the following: Differential privacy processing, data anonymization processing, homomorphic encryption, and secure multi-party computation.

4. A semantic communication method, characterized in that, Applied to the receiving end, including: Receive encrypted semantic information sent by the sender; Channel decoding is performed on the encrypted semantic information to obtain the first encrypted semantic information; A codec network structure based on a public knowledge base is used to semantically decode the first encrypted semantic information to obtain the second encrypted semantic information. When the encryption method is homomorphic encryption, the codec network structure is constructed as follows: the activation function of the codec network structure is set to a homomorphic operation function; the max pooling layer in the codec network structure is replaced with an average pooling layer. The second encrypted semantic information is decrypted to obtain the plaintext semantic information; A decoding network structure based on a private knowledge base is used to perform local semantic decoding on the plaintext semantic information to obtain semantic private decoding information.

5. A semantic communication device, characterized in that, Applied to the sending end, including: The encryption module is used to obtain encrypted data based on the received source signal; The first encoding module is used to perform semantic encoding on the encrypted data using an encoding / decoding network structure built on a public knowledge base to obtain an encrypted semantic vector. When the encryption method is homomorphic encryption, the encoding / decoding network structure is constructed as follows: the activation function of the encoding / decoding network structure is set to a homomorphic operation function; the max pooling layer in the encoding / decoding network structure is replaced with an average pooling layer. The second encoding module is used to perform channel encoding on the encrypted semantic vector to obtain encrypted semantic information. The sending module is used to send the encrypted semantic information to the receiving end; The step of obtaining encrypted data based on the received source signal includes: when the data volume of the source signal exceeds a target threshold, performing semantic extraction on the source signal to obtain the hidden layer semantics output by the intermediate layer of the encoding / decoding network structure; and encrypting the hidden layer semantics to obtain the encrypted data.

6. A semantic communication device, characterized in that, Applied to the receiving end, including: The receiving module is used to receive encrypted semantic information sent by the sending end; The first decoding module is used to perform channel decoding on the encrypted semantic information to obtain the first encrypted semantic information; The second decoding module is used to perform semantic decoding on the first encrypted semantic information using an encoding / decoding network structure built on a public knowledge base to obtain the second encrypted semantic information; when the encryption method is homomorphic encryption, the encoding / decoding network structure is constructed in the following way: setting the activation function of the encoding / decoding network structure as a homomorphic operation function; replacing the max pooling layer in the encoding / decoding network structure with an average pooling layer; The decryption module is used to decrypt the second encrypted semantic information to obtain plaintext semantic information; The third decoding module is used to perform local semantic decoding on the plaintext semantic information using a decoding network structure built on a private knowledge base, so as to obtain semantic private decoding information.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the semantic communication method as described in any one of claims 1 to 4.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the semantic communication method as described in any one of claims 1 to 4.

9. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the semantic communication method as described in any one of claims 1 to 4.

Images