A privacy-enhanced semantic communication method based on trusted execution environment secure inference

Abstract

The present disclosure relates to a privacy-enhanced semantic communication method based on trusted execution environment secure inference. The method comprises: performing a first model inference operation on to-be-transmitted data by using a TEE, decomposing model parameters into a first part and a second part, performing obfuscation processing on intermediate data according to a first obfuscation manner, inputting the obfuscated result into a rich execution environment (REE), controlling the REE to perform a linear operation based on the obfuscated result and the second part to obtain an obfuscated linear operation result, inputting the linear operation result into the TEE, controlling the TEE to determine a linear operation result before obfuscation based on the first obfuscation manner, the first part and the model parameters, and sending the semantic features obtained by the first model inference operation to a target receiver in the case that the first model inference operation is completed. The present scheme improves the confidentiality of data transmission.

Description

Technical Field

[0001] This disclosure relates to the field of wireless security technology, and in particular to a privacy-enhanced semantic communication method based on secure reasoning in a trusted execution environment. Background Technology

[0002] Among related technologies, semantic communication, as a novel communication paradigm, combines artificial intelligence with semantic information transmission rather than the communication symbols themselves, performing joint analysis and processing of the source and channel. This technology improves the transmission efficiency and capabilities of wireless communication systems, adapting to more complex and dynamic application scenarios, attracting widespread attention from academia and industry, and experiencing rapid development. Unlike syntactic communication, semantic communication is communication that extracts and transmits the information required for a specific communication task.

[0003] The semantic encoder is responsible for performing semantic analysis on the transmitted data and extracting semantic features; the channel encoder comprehensively analyzes the semantic features and channel conditions, and compresses the semantic features into a wireless signal suitable for transmission; the channel decoder restores the received signal to semantic features; and the semantic decoder analyzes the semantic features and outputs the result to be transmitted.

[0004] The communication process is divided into two phases: the model inference phase and the wireless transmission phase. The model inference phase refers to the various parts of the neural network performing corresponding operations on a given input; the wireless transmission phase refers to the transmission of the output wireless signal symbols of the channel encoder in the wireless channel. However, to deploy a semantic communication system on a large scale, the neural network model needs to be directly deployed on a massive number of nodes that need to communicate, which will cause leakage of neural network parameters. Summary of the Invention

[0005] To overcome the problems existing in related technologies, this disclosure provides a privacy-enhanced semantic communication method based on secure reasoning in a trusted execution environment.

[0006] According to a first aspect of the present disclosure, a privacy-enhanced semantic communication method based on secure reasoning in a trusted execution environment is provided, comprising:

[0007] Input the data to be transmitted into the Trusted Execution Environment (TEE);

[0008] The first model inference operation is performed on the data to be transmitted using the TEE;

[0009] When the first model inference operation requires determining the linear operation result for intermediate data x and the model parameter w, the intermediate data x is obfuscated according to the first obfuscation method to obtain the obfuscation result; the intermediate data is extracted from the data to be transmitted.

[0010] The obfuscation result is input into the rich execution environment (REE).

[0011] The REE is controlled to perform a linear operation based on the obfuscation result and the second part of the model parameter w, according to the linear calculation type corresponding to the linear operation result, to obtain the obfuscated linear operation result; the model parameter w includes a first part and a second part, which are obtained by decomposing the model parameter w in advance;

[0012] The obfuscated linear operation result is input into the TEE, and the TEE is controlled to determine the unobfuscated linear operation result based on the first obfuscation method, the first part, and the model parameter w.

[0013] When the first model inference operation is completed, the semantic features obtained from the first model inference operation are sent to the target receiver so that the target receiver can perform the second model inference operation based on the semantic features to obtain the target data; the operation mode of the second model inference operation is the same as that of the first model inference operation.

[0014] In some embodiments of this disclosure, the step of obfuscating x according to a first obfuscation method to obtain an obfuscation result includes:

[0015] Generate a random number n;

[0016] The random number n is added to the intermediate data x to obtain the confusion result.

[0017] In some embodiments of this disclosure, the tensor shape of the random number n is the same as the tensor shape of the intermediate data x.

[0018] In some embodiments of this disclosure, the step of sending the semantic features obtained from the first model inference operation to the target receiver after the first model inference operation is completed includes:

[0019] Once the first model inference operation is completed, the semantic features are encrypted using a key;

[0020] The encrypted semantic features are sent to the target recipient, so that the target recipient can use the key to decrypt the encrypted semantic features and obtain the semantic features.

[0021] In some embodiments of this disclosure, the step of sending the semantic features obtained from the first model inference operation to the target receiver after the first model inference operation is completed includes:

[0022] Once the first model inference operation is completed, the semantic features are input to the channel encoder to obtain the wireless signal output by the channel encoder.

[0023] The wireless signal is mapped onto a constellation diagram to obtain the coordinates of the symbol points corresponding to the wireless signal in the constellation diagram, and the coordinates of the symbol points are used as the quantized wireless signal to be transmitted.

[0024] The quantized wireless signal to be transmitted is sent to the target receiver.

[0025] In some embodiments of this disclosure, the first part and the second part are obtained by decomposing the model parameter w using the following formula:

[0026] w = a·b

[0027] Where a is the first part and b is the second part.

[0028] According to a second aspect of the present disclosure, a privacy-enhanced semantic communication apparatus based on secure reasoning in a trusted execution environment is provided, comprising:

[0029] The input unit is used to input the data to be transmitted into the Trusted Execution Environment (TEE).

[0030] A model inference unit is used to perform a first model inference operation on the data to be transmitted using the TEE.

[0031] The obfuscation unit is used to obfuscate the intermediate data x according to a first obfuscation method when the first model inference operation needs to determine the linear operation result for the intermediate data x and the model parameters w, so as to obtain an obfuscation result; the intermediate data is extracted from the data to be transmitted.

[0032] The input unit is also used to input the obfuscation result into the rich execution environment (REE);

[0033] A linear operation unit is used to control the REE to perform linear operations based on the obfuscation result and the second part of the model parameter w, according to the linear calculation type corresponding to the linear operation result, to obtain the obfuscated linear operation result; the model parameter w includes a first part and a second part, which are obtained by pre-decomposing the model parameter w;

[0034] A determining unit is used to input the obfuscated linear operation result into the TEE, and control the TEE to determine the linear operation result before obfuscation based on the first obfuscation method, the first part, and the model parameter w.

[0035] The sending unit is configured to send the semantic features obtained from the first model inference operation to the target receiver after the first model inference operation is completed, so that the target receiver can perform a second model inference operation based on the semantic features to obtain target data; the operation mode of the second model inference operation is the same as that of the first model inference operation.

[0036] According to a third aspect of the present disclosure, an electronic device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the method as described in any one of the first aspects.

[0037] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the method as described in any one of the first aspects.

[0038] According to a fifth aspect of the present disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the method as described in any one of the first aspects.

[0039] The technical solutions provided by the embodiments of this disclosure can include the following beneficial effects: The data to be transmitted is input into a Trusted Execution Environment (TEE); a first model inference operation is performed on the data to be transmitted using the TEE; when the first model inference operation requires determining the linear operation result for intermediate data x and model parameters w, the intermediate data x is obfuscated according to a first obfuscation method to obtain an obfuscated result; the intermediate data is extracted from the data to be transmitted; the obfuscated result is input into a Rich Execution Environment (REE); the REE is controlled to perform a linear operation based on the obfuscated result and a second part of the model parameters w, according to the linear calculation type corresponding to the linear operation result, to obtain an obfuscated linear operation result; the linear operation result is input into the TEE, and the TEE is controlled to determine the linear operation result before obfuscation based on the first obfuscation method, the first part, and the model parameters w; when the first model inference operation is completed, the semantic features obtained from the first model inference operation are sent to the target receiver. This utilizes a combination of TEE and REE to efficiently execute the first model inference operation while keeping the model parameters and intermediate results confidential during the inference process, effectively avoiding information leakage caused by semantic feature extraction of the data to be transmitted and improving the confidentiality of data transmission.

[0040] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0041] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0042] Figure 1 This is a flowchart illustrating a privacy-enhanced semantic communication method based on secure reasoning in a trusted execution environment, according to an exemplary embodiment.

[0043] Figure 2 This is a schematic diagram illustrating an existing data transmission method according to an exemplary embodiment.

[0044] Figure 3 This is a schematic diagram illustrating a model reasoning process according to an exemplary embodiment.

[0045] Figure 4 This is a data transmission flowchart illustrated according to an exemplary embodiment.

[0046] Figure 5 This is a block diagram illustrating a privacy-enhanced semantic communication device based on secure reasoning in a trusted execution environment, according to an exemplary embodiment.

[0047] Figure 6 This is a block diagram illustrating an apparatus for a privacy-enhanced semantic communication method for secure reasoning based on a trusted execution environment, according to an exemplary embodiment. Detailed Implementation

[0048] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the invention as detailed in the appended claims.

[0049] The terminology used in this disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the present disclosure. The singular forms “a” and “the” as used in this disclosure and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise.

[0050] It should be understood that although the terms first, second, third, etc., may be used to describe various information in embodiments of this disclosure, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first information may also be referred to as second information without departing from the scope of embodiments of this disclosure, and similarly, second information may also be referred to as first information. Depending on the context, the words “if” and “suppose” as used herein may be interpreted as “when”, “when”, or “in response to a determination”.

[0051] Furthermore, various forms of processes shown in the embodiments of this disclosure can be used to reorder, add, or delete steps. For example, the steps described in this application can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and no limitation is imposed herein.

[0052] Among related technologies, semantic communication, as a novel communication paradigm, combines artificial intelligence with semantic information transmission rather than the communication symbols themselves, performing joint analysis and processing of the source and channel. This technology improves the transmission efficiency and capabilities of wireless communication systems, adapting to more complex and dynamic application scenarios, attracting widespread attention from academia and industry, and experiencing rapid development. Unlike syntactic communication, semantic communication is communication that extracts and transmits the information required for a specific communication task.

[0053] See Appendix for semantic communication system architecture Figure 2 An end-to-end semantic communication system consists of a semantic codec and a channel codec. Both are implemented using deep learning neural networks.

[0054] The semantic encoder is responsible for performing semantic analysis on the transmitted data and extracting semantic features; the channel encoder comprehensively analyzes the semantic features and channel conditions, and compresses the semantic features into a wireless signal suitable for transmission; the channel decoder restores the received signal to semantic features; and the semantic decoder analyzes the semantic features and outputs the result to be transmitted.

[0055] The communication process is divided into two stages: the model inference stage and the wireless transmission stage. The model inference stage refers to the various parts of the neural network performing corresponding operations on the given input; the wireless transmission stage refers to the transmission of the output wireless signal symbols of the channel encoder in the wireless channel.

[0056] It's important to note that large-scale deployment of semantic communication systems requires directly deploying the neural network model on a massive number of nodes that need to communicate. This leads to the leakage of neural network parameters, and vendors providing closed-source models are unwilling to disclose their model parameters.

[0057] Furthermore, the leakage of neural network parameters and intermediate inference results facilitates white-box attacks by attackers. White-box attacks in neural network attacks refer to attackers having complete knowledge of the target model's internal structure, parameters, and training data, and using this information for eavesdropping (obtaining transmitted data), model theft (obtaining model parameters), and model reverse engineering (obtaining information about the training dataset).

[0058] Furthermore, the open nature of wireless communication allows attackers to easily obtain transmitted wireless signals. Semantic communication transmits wireless signals containing a wealth of information, aiding attackers in their attacks.

[0059] Furthermore, while methods such as multi-party secure computation and homomorphic encryption can protect model parameters from leakage, they incur excessive computational overhead.

[0060] To address the aforementioned issues, this disclosure provides a privacy-enhanced semantic communication method based on secure reasoning within a Trusted Execution Environment (TEE). This method involves: inputting the data to be transmitted into the TEE; performing a first model inference operation on the data to be transmitted using the TEE; decomposing the model parameters w into a first part and a second part using the TEE; when the first model inference operation requires determining the linear operation result for intermediate data x and model parameters w, obfuscating the intermediate data x according to a first obfuscation method to obtain an obfuscated result; the intermediate data being extracted from the data to be transmitted; inputting the obfuscated result and the second part into a Rich Execution Environment (REE); and controlling the REE based on the obfuscated result and the second part. The second part involves performing linear operations according to the linear calculation type corresponding to the linear operation result, obtaining the obfuscated linear operation result; inputting the linear operation result into the TEE, controlling the TEE to determine the linear operation result before obfuscation based on the first obfuscation method, the first part, and the model parameter w; after the first model inference operation is completed, sending the semantic features obtained from the first model inference operation to the target receiver, thereby using the combination of TEE and REE to efficiently execute the first model inference operation while keeping the model parameters and intermediate results during the inference process confidential, effectively avoiding information leakage caused by semantic feature extraction of the data to be transmitted, and improving the confidentiality of data transmission.

[0061] Figure 1 This is a flowchart illustrating a privacy-enhanced semantic communication method based on secure reasoning in a trusted execution environment, according to an exemplary embodiment. Figure 1 As shown, it should be noted that the privacy-enhanced semantic communication method based on secure reasoning in a trusted execution environment, as described in this disclosure, is applied in a privacy-enhanced semantic communication device based on secure reasoning in a trusted execution environment. Figure 1 As shown, the method may include the following steps:

[0062] Step 101: Input the data to be transmitted into the Trusted Execution Environment (TEE).

[0063] It should be noted that TEE is a hardware-isolated secure execution environment that protects the confidentiality and integrity of sensitive data.

[0064] In one embodiment, steps 101 to 107 can be performed using a pre-trained neural network model.

[0065] As an example, a TEE can be configured on all communication nodes, and users on each communication node can use mature technologies to securely transmit the model to the TEE.

[0066] Step 102: Utilize the TEE to perform the first model inference operation on the data to be transmitted.

[0067] Understandably, the purpose of model inference is to extract the necessary semantic features from the data to be transmitted.

[0068] Step 103: If the first model inference operation needs to determine the linear operation result for intermediate data x and model parameters w, the intermediate data x is obfuscated according to the first obfuscation method to obtain the obfuscation result.

[0069] Intermediate data is extracted from the data to be transmitted.

[0070] It should be noted that the model inference process requires multiple linear and nonlinear calculations, with the nonlinear calculations being performed directly by the TEE. Furthermore, the intermediate data x and model parameters w mentioned above can be data associated with any linear operation; that is, whenever a linear operation is required, obfuscation is performed so that the REE can perform linear calculations based on the obfuscation results.

[0071] In one embodiment, the TEE first encrypts and obfuscates the data to be unloaded, then the REE performs linear calculations, and finally the TEE recovers the required calculation results based on the linear calculation results. In this way, even if the data in the REE is leaked, the attacker cannot obtain the real w and x.

[0072] In some embodiments of this disclosure, step 104 may specifically include the following steps:

[0073] Generate a random number n;

[0074] Add the random number n to the intermediate data x to obtain the scrambled result.

[0075] In some embodiments of this disclosure, the tensor shape of the random number n is the same as the tensor shape of the intermediate data x.

[0076] Step 104: Input the obfuscation results into the Rich Execution Environment (REE).

[0077] It should be noted that REE refers to a conventional operating system environment, which has low security but more powerful computing capabilities.

[0078] In one embodiment, the model parameters w, intermediate results x, split values ​​a, and random numbers n are stored in the TEE, while the REE contains only obfuscated b and x+n. Even if the data in the REE is leaked, the attacker cannot obtain the real w and x.

[0079] Step 105: Control the REE to perform linear operations based on the second part of the obfuscation result and model parameter w, according to the linear calculation type corresponding to the linear operation result, to obtain the obfuscated linear operation result.

[0080] The model parameter w includes a first part and a second part, which are obtained by decomposing the model parameter w in advance.

[0081] As an example, the linear computation type mentioned above can be matrix multiplication or convolution.

[0082] In one possible embodiment, the second part corresponding to each of the different model parameters and the corresponding linear calculation type can be pre-stored in the REE.

[0083] In one embodiment, for a linear computation w×x, by offloading this linear computation to the REE, the TEE only needs to calculate a·rw×n, where w×n can be calculated in advance since it is irrelevant to intermediate results. Therefore, during real-time inference, the TEE, which originally needed to calculate a matrix multiplication, only needs to calculate a matrix scalar multiplication and a matrix subtraction in this solution, effectively improving the computational efficiency of the TEE and thus improving the efficiency of model inference.

[0084] In one embodiment, the model parameter w can be the model parameter of a convolutional neural network model.

[0085] In some embodiments of this disclosure, the first part and the second part are obtained by decomposing the model parameter w using the following formula:

[0086] w = a·b

[0087] Where a is the first part and b is the second part.

[0088] In one embodiment, to reduce the complexity of model inference, the model parameters are decomposed so that the REE can perform linear operations on a portion of them, effectively reducing the amount of computation required by the TEE during real-time inference.

[0089] Step 106: Input the obfuscated linear operation result into the TEE, and control the TEE to determine the linear operation result before obfuscation based on the first obfuscation method, the first part, and the model parameter w.

[0090] In one embodiment, the TEE stores a first obfuscation method, a first part, and model parameters w, so the result of the linear operation before obfuscation can be derived by inverse calculation based on the linear operation and the above information.

[0091] For example, such as Figure 3 As shown, after receiving the data x+n, the REE performs a linear operation r = b × (x+n) and sends r to the TEE. The TEE uses the secretly stored a, w, and n to calculate a·rw × n, which is the required w × x, because:

[0092] a·rw×n=a·b×(x+n)-w×n=c×b×(x+n)-w×n=w×(x+n)-w×n

[0093] =w×x

[0094] Where c satisfies c×b=a·b=w.

[0095] For example, when the operation · represents scalar multiplication and the operation × represents matrix multiplication, a is a scalar and b is a matrix. c = aI, where I is the identity matrix, and this condition holds.

[0096] Step 107: After the first model inference operation is completed, the semantic features obtained from the first model inference operation are sent to the target receiver so that the target receiver can perform the second model inference operation based on the semantic features to obtain the target data.

[0097] The operation method of the second model reasoning operation is the same as that of the first model reasoning operation.

[0098] In some embodiments of this disclosure, after the target receiver receives the semantic feature signal sent by the sender, it needs to sequentially use a channel decoder to recover the signal into semantic features, and then use a semantic decoder to further perform semantic analysis on the semantic features. That is, the channel decoder and semantic decoder are used to perform a second model inference operation in the same manner as the first model inference operation, thereby obtaining the target data. The second model inference operation includes the following steps:

[0099] Step a1: In the case that the second model inference operation needs to determine the linear operation result for intermediate data x and model parameters w, the intermediate data x is obfuscated according to the first obfuscation method to obtain the obfuscation result; the intermediate data is extracted from the data to be transmitted.

[0100] Step a2: Input the obfuscation result into the rich execution environment REE;

[0101] Step a3: Control the REE to perform linear operations based on the obfuscation result and the second part of the model parameter w, according to the linear calculation type corresponding to the linear operation result, to obtain the obfuscated linear operation result; the model parameter w includes a first part and a second part, which are obtained by decomposing the model parameter w in advance;

[0102] Step a4: Input the obfuscated linear operation result into the TEE, and control the TEE to determine the linear operation result before obfuscation based on the first obfuscation method, the first part, and the model parameter w.

[0103] It should be noted that the specific implementation of steps a1-a4 above can be achieved by any of the implementation methods disclosed in this disclosure.

[0104] In some embodiments of this disclosure, step 107 may specifically include the following steps:

[0105] Once the first model inference operation is completed, the semantic features are encrypted using a key.

[0106] The encrypted semantic features are sent to the target recipient, who then uses a key to decrypt the encrypted semantic features and obtain the semantic features.

[0107] In some embodiments of this disclosure, the sender and receiver of data can pre-negotiate a key p to obtain the key. An attacker can only obtain the encrypted wireless signal but cannot decrypt it.

[0108] In some embodiments of this disclosure, step 107 may specifically include the following steps:

[0109] Once the first model inference operation is completed, the semantic features are input into the channel encoder to obtain the wireless signal output by the channel encoder.

[0110] The wireless signal is mapped onto the constellation diagram to obtain the coordinates of the symbol points corresponding to the wireless signal in the constellation diagram. The coordinates of the symbol points are then used as the quantized wireless signal to be transmitted.

[0111] The quantized wireless signal to be transmitted is sent to the target receiver.

[0112] In one embodiment, after the first model inference operation is completed, the semantic features are input to the channel encoder, which comprehensively analyzes the semantic features and the channel conditions, and compresses the semantic features into a wireless signal suitable for transmission.

[0113] It should be noted that in communication principles, a constellation diagram is a graphical tool used to represent digitally modulated signals. It visually displays the distribution of signals on the complex plane (IQ plane), with each point corresponding to a specific symbol used to transmit digital information.

[0114] In one embodiment, a signal symbol can be mapped to its nearest symbol point in the constellation diagram to obtain the symbol point coordinates, which are then used as the quantized wireless signal to be transmitted. The constellation diagram is designed using machine learning. The forward propagation process of model training simulates the above mapping process and adds corresponding training objectives, such as minimizing the distance between the channel encoder output symbol and its nearest symbol point in the constellation diagram; minimizing the loss function of the semantic communication model task; minimizing the information entropy of the signal symbol, etc.

[0115] Furthermore, various methods can be used to simplify wireless signal symbols and achieve constellation diagrams with a limited number of symbol points. One approach is through quantization; in addition to quantization of single scalars, such as 8-bit or 16-bit quantization, overall vector quantization can also be performed.

[0116] In one embodiment, for the quantized wireless signal Sig to be transmitted, the sender encrypts it based on key p to obtain Enc. p (Sig).

[0117] In this embodiment of the disclosure, by utilizing a quantization mechanism, the semantic communication system is operated on a constellation diagram with a finite number of symbol points, which can significantly reduce the requirements for the hardware precision of the communication equipment. In addition, the transmission results can eliminate interference from the wireless channel, so as to enable real-time encryption and decryption schemes.

[0118] In some embodiments of this disclosure, such as Figure 4As shown, when the sender needs to transmit data for communication, the semantic encoder performs model inference as proposed in this disclosure to obtain semantic features. The channel encoder converts the semantic features into wireless signals, quantizes the wireless signals according to the quantization processing method proposed in this disclosure, encrypts them, and transmits the encrypted data to the receiver via the wireless channel. The receiver decrypts the received data, then decodes the data using the channel decoder and performs semantic decoding using the semantic decoder to obtain the data to be sent by the sender. Since the receiver also needs to send data to other parties, the receiver can also install a TEE, thus using a combination of TEE and REE for model inference.

[0119] According to the privacy-enhanced semantic communication method based on secure reasoning in a Trusted Execution Environment (TEE) proposed in this disclosure, the method involves: inputting the data to be transmitted into the TEE; performing a first model reasoning operation on the data to be transmitted using the TEE; decomposing the model parameter w into a first part and a second part using the TEE; when the first model reasoning operation requires determining the linear operation result for intermediate data x and model parameter w, obfuscating the intermediate data x according to a first obfuscation method to obtain an obfuscated result; the intermediate data being extracted from the data to be transmitted; inputting the obfuscated result and the second part into a Rich Execution Environment (REE); controlling the REE to perform linear operations based on the obfuscated result and the second part, according to the linear calculation type corresponding to the linear operation result, to obtain an obfuscated linear operation result; inputting the linear operation result into the TEE, controlling the TEE to determine the linear operation result before obfuscation based on the first obfuscation method, the first part, and the model parameter w; and, upon completion of the first model reasoning operation, sending the semantic features obtained from the first model reasoning operation to the target receiver. This method utilizes a combination of TEE and REE to efficiently execute the first model reasoning operation while keeping the model parameters and intermediate results confidential during the reasoning process, effectively avoiding information leakage caused by semantic feature extraction of the data to be transmitted and improving the confidentiality of data transmission.

[0120] Figure 5 This is a block diagram illustrating a privacy-enhanced semantic communication device based on secure reasoning in a trusted execution environment, according to an exemplary embodiment. (Refer to...) Figure 5 The device includes an input unit 501, a model reasoning unit 502, a confusion unit 503, an input unit 504, a linear operation unit 505, a determination unit 506, and a sending unit 507.

[0121] The input unit 501 is used to input the data to be transmitted into the Trusted Execution Environment (TEE).

[0122] The model inference unit 502 is used to perform a first model inference operation on the data to be transmitted using the TEE;

[0123] The obfuscation unit 503 is used to obfuscate the intermediate data x according to a first obfuscation method when the first model inference operation needs to determine the linear operation result for the intermediate data x and the model parameters w, so as to obtain the obfuscation result; the intermediate data is extracted from the data to be transmitted.

[0124] The input unit 504 is also used to input the obfuscation result and the second part into the rich execution environment REE;

[0125] The linear operation unit 505 is used to control the REE to perform linear operations based on the obfuscation result and the second part, according to the linear calculation type corresponding to the linear operation result, to obtain the obfuscated linear operation result.

[0126] The determining unit 506 is used to input the obfuscated linear operation result into the TEE and control the TEE to determine the linear operation result before obfuscation based on the first obfuscation method, the first part and the model parameter w.

[0127] The sending unit 507 is used to send the semantic features obtained from the first model inference operation to the target receiver after the first model inference operation is completed, so that the target receiver can perform the second model inference operation based on the semantic features to obtain the target data; the operation mode of the second model inference operation is the same as that of the first model inference operation.

[0128] In some embodiments of this disclosure, the obfuscation unit 503 may specifically be used for:

[0129] Generate a random number n;

[0130] Add the random number n to the intermediate data x to obtain the scrambled result.

[0131] In some embodiments of this disclosure, the tensor shape of the random number n is the same as the tensor shape of the intermediate data x.

[0132] In some embodiments of this disclosure, the sending unit 507 may specifically be used for:

[0133] Once the first model inference operation is completed, the semantic features are encrypted using a key.

[0134] The encrypted semantic features are sent to the target recipient, who then uses a key to decrypt the encrypted semantic features and obtain the semantic features.

[0135] In some embodiments of this disclosure, the sending unit 507 may specifically be used for:

[0136] Once the first model inference operation is completed, the semantic features are input into the channel encoder to obtain the wireless signal output by the channel encoder.

[0137] The wireless signal is mapped onto the constellation diagram to obtain the coordinates of the symbol points corresponding to the wireless signal in the constellation diagram. The coordinates of the symbol points are then used as the quantized wireless signal to be transmitted.

[0138] The quantized wireless signal to be transmitted is sent to the target receiver.

[0139] In some embodiments of this disclosure, the first part and the second part are obtained by decomposing the model parameter w using the following formula:

[0140] w = a·b

[0141] Where a is the first part and b is the second part.

[0142] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0143] According to the privacy-enhanced semantic communication device based on secure reasoning in a Trusted Execution Environment (TEE) proposed in this disclosure, the device performs the following steps: First, it inputs the data to be transmitted into the TEE; it performs a first model reasoning operation on the data to be transmitted using the TEE; it decomposes the model parameter w into a first part and a second part using the TEE; when the first model reasoning operation requires determining the linear operation result for intermediate data x and model parameter w, it obfuscates the intermediate data x according to a first obfuscation method to obtain an obfuscated result; the intermediate data is extracted from the data to be transmitted; it inputs the obfuscated result and the second part into a Rich Execution Environment (REE); it controls the REE to perform linear operations based on the obfuscated result and the second part, according to the linear calculation type corresponding to the linear operation result, to obtain an obfuscated linear operation result; it inputs the linear operation result into the TEE, and controls the TEE to determine the linear operation result before obfuscation based on the first obfuscation method, the first part, and the model parameter w; when the first model reasoning operation is completed, it sends the semantic features obtained from the first model reasoning operation to the target receiver. This method, combining the TEE and the REE, efficiently executes the first model reasoning operation while keeping the model parameters and intermediate results confidential, effectively avoiding information leakage caused by semantic feature extraction of the data to be transmitted and improving the confidentiality of data transmission.

[0144] Figure 6This is a block diagram illustrating an apparatus for a privacy-enhanced semantic communication method based on secure reasoning in a trusted execution environment, according to an exemplary embodiment. For example, apparatus 600 may be an electronic device, such as a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness equipment, personal digital assistant, etc.

[0145] Reference Figure 6 The device 600 may include one or more of the following components: a processing component 602, a memory 604, a power component 606, a multimedia component 608, an audio component 610, an input / output (I / O) interface 612, a sensor component 614, and a communication component 616.

[0146] Processing component 602 typically controls the overall operation of device 600, such as operations associated with display, telephone calls, data communication, camera operation, and recording. Processing component 602 may include one or more processors 620 to execute instructions to perform all or part of the steps of the methods described above. Furthermore, processing component 602 may include one or more modules to facilitate interaction between processing component 602 and other components. For example, processing component 602 may include a multimedia module to facilitate interaction between multimedia component 608 and processing component 602.

[0147] Memory 604 is configured to store various types of data to support the operation of device 600. Examples of this data include instructions for any application or method operating on device 600, contact data, phonebook data, messages, pictures, videos, etc. Memory 604 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0148] The power supply component 606 provides power to the various components of the device 600. The power supply component 606 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power to the device 600.

[0149] Multimedia component 608 includes a screen that provides an output interface between the device 600 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 608 includes a front-facing camera and / or a rear-facing camera. When the device 600 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0150] Audio component 610 is configured to output and / or input audio signals. For example, audio component 610 includes a microphone (MIC) configured to receive external audio signals when device 600 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 604 or transmitted via communication component 616. In some embodiments, audio component 610 also includes a speaker for outputting audio signals.

[0151] I / O interface 612 provides an interface between processing component 602 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0152] Sensor assembly 614 includes one or more sensors for providing status assessments of various aspects of device 600. For example, sensor assembly 614 may detect the on / off state of device 600, the relative positioning of components such as the display and keypad of device 600, changes in the position of device 600 or a component of device 600, the presence or absence of user contact with device 600, the orientation or acceleration / deceleration of device 600, and temperature changes of device 600. Sensor assembly 614 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 614 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 614 may also include an accelerometer, a gyroscope, a magnetometer, a pressure sensor, or a temperature sensor.

[0153] Communication component 616 is configured to facilitate wired or wireless communication between device 600 and other devices. Device 600 can access wireless networks based on communication standards, such as WiFi, 2G, or 3G, or combinations thereof. In one exemplary embodiment, communication component 616 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 616 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0154] In an exemplary embodiment, the apparatus 600 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.

[0155] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 604 including instructions, which can be executed by a processor 620 of the device 600 to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0156] In an exemplary embodiment, a computer program product is also provided, including a computer program that implements the above-described method when executed by a processor 620 of the device 600.

[0157] Other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of the invention are indicated by the following claims.

[0158] It should be understood that the present invention is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A privacy-enhanced semantic communication method based on secure reasoning in a trusted execution environment, characterized in that, include: Input the data to be transmitted into the Trusted Execution Environment (TEE); The first model inference operation is performed on the data to be transmitted using the TEE; The first model inference operation needs to determine the intermediate data. and model parameters In the case of linear operation results, the intermediate data x is obfuscated according to the first obfuscation method to obtain the obfuscation result; The intermediate data is extracted from the data to be transmitted; The obfuscation result is input into the rich execution environment (REE). The REE is controlled based on the obfuscation result and the model parameters. The second part involves performing a linear operation according to the linear calculation type corresponding to the linear operation result, to obtain the obfuscated linear operation result; the model parameters It includes a first part and a second part, which are pre-configured model parameters. Obtained through disassembly; The obfuscated linear operation result is input into the TEE, and the TEE is controlled based on the first obfuscation method, the first part, and the model parameters. Determine the result of the linear operation before confusion; Upon completion of the first model inference operation, the semantic features obtained from the first model inference operation are sent to the target receiver, so that the target receiver can perform the second model inference operation based on the semantic features to obtain the target data. The operation method of the second model inference operation is the same as that of the first model inference operation.

2. The privacy-enhanced semantic communication method based on secure reasoning in a trusted execution environment according to claim 1, characterized in that, The obfuscation of x according to the first obfuscation method to obtain the obfuscation result includes: Generate a random number n; The random number n is added to the intermediate data x to obtain the confusion result.

3. The privacy-enhanced semantic communication method based on secure reasoning in a trusted execution environment according to claim 2, characterized in that, The tensor shape of the random number n is the same as the tensor shape of the intermediate data x.

4. The privacy-enhanced semantic communication method based on secure reasoning in a trusted execution environment according to claim 1, characterized in that, The step of sending the semantic features obtained from the first model inference operation to the target receiver after the first model inference operation is completed includes: Once the first model inference operation is completed, the semantic features are encrypted using a key; The encrypted semantic features are sent to the target recipient, so that the target recipient can use the key to decrypt the encrypted semantic features and obtain the semantic features.

5. The privacy-enhanced semantic communication method based on secure reasoning in a trusted execution environment according to claim 1, characterized in that, The step of sending the semantic features obtained from the first model inference operation to the target receiver after the first model inference operation is completed includes: Once the first model inference operation is completed, the semantic features are input to the channel encoder to obtain the wireless signal output by the channel encoder. The wireless signal is mapped onto a constellation diagram to obtain the coordinates of the symbol points corresponding to the wireless signal in the constellation diagram, and the coordinates of the symbol points are used as the quantized wireless signal to be transmitted. The quantized wireless signal to be transmitted is sent to the target receiver.

6. The privacy-enhanced semantic communication method based on secure reasoning in a trusted execution environment according to claim 1, characterized in that, The first and second parts are determined by the following formulas for the model parameters. The following was obtained through disassembly: Where a represents the first part and b represents the second part. For scalar multiplication.

7. A privacy-enhanced semantic communication device based on secure reasoning in a trusted execution environment, characterized in that, include: The input unit is used to input the data to be transmitted into the Trusted Execution Environment (TEE). A model inference unit is used to perform a first model inference operation on the data to be transmitted using the TEE. The obfuscation unit is used to determine the intermediate data during the first model inference operation. and model parameters In the case of a linear operation result, the intermediate data x is obfuscated according to a first obfuscation method to obtain an obfuscation result; the intermediate data is extracted from the data to be transmitted. The input unit is also used to input the obfuscation result into the rich execution environment (REE); Linear operation unit, used to control the REE based on the confusion result and the model parameters. The second part involves performing a linear operation according to the linear calculation type corresponding to the linear operation result, to obtain the obfuscated linear operation result; the model parameters It includes a first part and a second part, which are pre-configured model parameters. Obtained through disassembly; The determining unit is used to input the obfuscated linear operation result into the TEE, and control the TEE based on the first obfuscation method, the first part, and the model parameters. Determine the result of the linear operation before confusion; The sending unit is configured to send the semantic features obtained from the first model inference operation to the target receiver after the first model inference operation is completed, so that the target receiver can perform a second model inference operation based on the semantic features to obtain target data. The operation method of the second model inference operation is the same as that of the first model inference operation.

8. An electronic device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, The computer program, when executed by a processor, implements the method as described in any one of claims 1 to 6.

Images