Semantic communication method, device and system for electric power Internet of Things terminal

Abstract

The invention provides a semantic communication method, device and system for a power internet of things terminal, and belongs to the field of semantic communication. The method comprises the following steps: receiving semantic model parameters of a target semantic model obtained by training of a cloud platform; updating a local semantic model based on the semantic model parameters, and encoding the to-be-transmitted image data according to the updated local semantic model to obtain to-be-transmitted semantic features; the local semantic model and the target semantic model have the same structure; performing data confusion on the row dimension, the column dimension and the channel dimension of the to-be-transmitted semantic feature in sequence, recording a data confusion scheme, and obtaining a confused semantic feature; and encrypting the data confusion scheme and the confusion semantic features and then transmitting the data confusion scheme and the confusion semantic features to a target power Internet of Things terminal. According to the method, the energy consumption and the time consumption required by the electric power Internet of Things terminal for realizing semantic communication can be reduced, and the semantic communication safety is ensured through data confusion.

Description

Technical Field

[0001] The present invention relates to the field of semantic communication technology, and in particular to a semantic communication method, device and system for power Internet of Things terminals. Background Art

[0002] With the rapid development of digital society, wireless communication technology is transitioning from 5G technology to 6G technology. Compared with 5G, the future 6G communication network will need to support more devices and complex intelligent application scenarios. However, the existing communication system mainly focuses on the transmission of bit data, which is difficult to meet the future needs for understanding the meaning and intent of data. Therefore, semantic communication technology came into being.

[0003] Semantic communication aims to improve the efficiency and reliability of communication systems by understanding and transmitting the true meaning of information (i.e., semantics) rather than just transmitting raw data. However, implementing semantic communication on resource-constrained IoT terminals, such as power IoT terminals, faces many challenges, such as the large amount of computing resources consumed by model training and security risks in the data transmission process. Therefore, a new solution is needed to realize the application of semantic communication technology on IoT terminals. Summary of the invention

[0004] The embodiments of the present invention provide a semantic communication method, device and system for power Internet of Things terminals to solve the problem that the existing semantic communication methods are not suitable for application on Internet of Things terminals.

[0005] In a first aspect, an embodiment of the present invention provides a semantic communication method for a power Internet of Things terminal, comprising:

[0006] Receive semantic model parameters of the target semantic model trained by the cloud platform;

[0007] The local semantic model is updated based on the semantic model parameters, and the image data to be transmitted is encoded according to the updated local semantic model to obtain the semantic features to be transmitted; the local semantic model and the target semantic model have the same structure;

[0008] Sequentially performing data obfuscation on the row dimension, column dimension, and channel dimension of the semantic feature to be transmitted, recording the data obfuscation scheme and obtaining the obfuscated semantic feature;

[0009] The data obfuscation scheme and the obfuscated semantic features are encrypted and transmitted to the target power Internet of Things terminal.

[0010] In a possible implementation, the data obfuscation scheme includes a row data obfuscation scheme, a column data obfuscation scheme, and a channel data obfuscation scheme;

[0011] Sequentially performing data obfuscation on the row dimension, column dimension, and channel dimension of the semantic feature to be transmitted, recording the data obfuscation scheme and obtaining the obfuscated semantic feature, including:

[0012] Randomly arrange the row dimensions of the semantic features to be transmitted to obtain row obfuscated semantic features, and record the corresponding random arrangement results as a row data obfuscation scheme;

[0013] Randomly arranging the column dimensions of the row confusion semantic features to obtain column confusion semantic features, and recording the corresponding random arrangement results as a column data obfuscation scheme;

[0014] The channel dimensions of the column obfuscated semantic features are randomly arranged to obtain obfuscated semantic features, and the corresponding random arrangement results are recorded as a channel data obfuscation scheme.

[0015] In a possible implementation, randomly arranging the row dimensions of the semantic features to be transmitted to obtain row obfuscated semantic features, and recording the corresponding random arrangement results as a row data obfuscation scheme includes:

[0016] Obtaining an array consisting of row index values ​​corresponding to each row of features in the semantic features to be transmitted, recorded as an original row index array;

[0017] Randomly arrange each row index value in the original row index array, and record the random arrangement result of the original row index array as a row data obfuscation scheme;

[0018] The order of each row feature in the semantic features to be transmitted is adjusted according to the order of each row index value in the row data obfuscation scheme to obtain a row obfuscated semantic feature.

[0019] In a possible implementation, randomly arranging the column dimension of the row confusion semantic feature to obtain the column confusion semantic feature, and recording the corresponding random arrangement result as a column data obfuscation scheme includes:

[0020] Obtaining an array consisting of column index values ​​corresponding to each column feature in the row confusion semantic feature, recorded as an original column index array;

[0021] Randomly arrange the column index values ​​in the original column index array, and record the random arrangement result of the original column index array as a column data obfuscation scheme;

[0022] The order of each column feature in the row confusion semantic feature is adjusted according to the order of each column index value in the column data obfuscation scheme to obtain a column confusion semantic feature.

[0023] In a possible implementation, randomly arranging the channel dimensions of the column obfuscated semantic features to obtain obfuscated semantic features, and recording the corresponding random arrangement results as a channel data obfuscation scheme, including:

[0024] Obtain an array consisting of channel index values ​​corresponding to each channel feature in the column confusion semantic feature, recorded as an original channel index array;

[0025] Randomly arrange each channel index value in the original channel index array, and record the random arrangement result of the original channel index array as a channel data obfuscation scheme;

[0026] The order of each channel feature in the column obfuscation semantic feature is adjusted according to the order of each channel index value in the channel data obfuscation scheme to obtain the obfuscated semantic feature.

[0027] In a possible implementation, the process of obtaining the semantic model parameters of the target semantic model includes:

[0028] Receive training data sent by the power Internet of Things terminal;

[0029] Input the image in the training data as the original image into the semantic encoder based on Swin Transformer, and obtain the reconstructed image output by the corresponding semantic decoder based on Swin Transformer;

[0030] Calculating a mean square error between each of the reconstructed images and the corresponding original image, and obtaining a loss function according to the mean square error;

[0031] Based on the Adam optimizer, training is performed with the minimum loss function as the objective function to obtain the semantic model parameters of the target semantic model.

[0032] In a possible implementation, the loss function is:

[0033]

[0034] in, is the loss function, i=1,2,…,N, N is the number of images in the training data, s i is the i-th original image, is the i-th reconstructed image, is the mean square error between the i-th reconstructed image and the i-th original image.

[0035] In a possible implementation, the objective function is:

[0036]

[0037] Wherein, α and β represent the weight and bias parameters of the neural network constituting the target semantic model, respectively. is the original image s and the reconstructed image The joint probability distribution of Represents the loss function In the joint probability distribution Seek hope.

[0038] In a second aspect, an embodiment of the present invention provides a semantic communication device for a power Internet of Things terminal, including:

[0039] A receiving module, used for receiving semantic model parameters of a target semantic model trained by a cloud platform;

[0040] A semantic encoding module, used for updating the local semantic model based on the semantic model parameters, and encoding the image data to be transmitted according to the updated local semantic model to obtain semantic features to be transmitted; the local semantic model and the target semantic model have the same structure;

[0041] A data obfuscation module, used to sequentially perform data obfuscation on the row dimension, column dimension and channel dimension of the semantic feature to be transmitted, record the data obfuscation scheme and obtain the obfuscated semantic feature;

[0042] The transmission module is used to encrypt the data obfuscation scheme and the obfuscated semantic features and transmit them to the target power Internet of Things terminal.

[0043] In a third aspect, an embodiment of the present invention provides a semantic communication system for power Internet of Things terminals, including: a cloud platform, a sending power Internet of Things terminal and a target power Internet of Things terminal;

[0044] The cloud platform is used to receive the training data sent by the sending power Internet of Things terminal or the target power Internet of Things terminal; input the image in the training data as the original image into the semantic encoder based on Swin Transformer, and obtain the corresponding reconstructed image output by the semantic decoder based on Swin Transformer; calculate the mean square error between each of the reconstructed images and the corresponding original image, and obtain the loss function according to the mean square error; based on the Adam optimizer, train with the minimum loss function as the objective function to obtain the target semantic model and the semantic model parameters of the target semantic model;

[0045] The sending power Internet of Things terminal is used to receive the semantic model parameters of the target semantic model trained by the cloud platform; update the local semantic model based on the semantic model parameters, and encode the image data to be transmitted according to the updated local semantic model to obtain the semantic features to be transmitted; the local semantic model and the target semantic model have the same structure; sequentially perform data obfuscation on the row dimension, column dimension and channel dimension of the semantic features to be transmitted, record the data obfuscation scheme and obtain the obfuscated semantic features; encrypt the data obfuscation scheme and the obfuscated semantic features and transmit them to the target power Internet of Things terminal;

[0046] The target power Internet of Things terminal is used to receive the semantic model parameters of the target semantic model trained by the cloud platform, the encrypted data obfuscation scheme and the encrypted obfuscated semantic features; decrypt the encrypted data obfuscation scheme and the encrypted obfuscated semantic features to obtain the data obfuscation scheme and the obfuscated semantic features; deobfuscate the obfuscated semantic features based on the data obfuscation scheme to obtain the semantic features to be transmitted; update the local semantic model based on the semantic model parameters, and decode the semantic features to be transmitted according to the updated local semantic model to obtain a reconstructed image of the image to be transmitted.

[0047] The embodiment of the present invention provides a semantic communication method, device and system for power Internet of Things terminals, which completes the training of the target semantic model on the cloud platform and transmits the model parameters back to the power Internet of Things terminal by receiving the semantic model parameters of the target semantic model trained by the cloud platform and updating the local semantic model based on the semantic model parameters, thereby reducing the energy consumption and time consumption of the power Internet of Things terminal. On this basis, the image data to be transmitted is encoded according to the updated local semantic model to obtain the semantic features to be transmitted, and then the row dimension, column dimension and channel dimension of the semantic features to be transmitted are sequentially obfuscated, the data obfuscation scheme is recorded and the obfuscated semantic features are obtained, and finally the data obfuscation scheme and the obfuscated semantic features are encrypted and transmitted to the target power Internet of Things terminal, thereby avoiding the situation where the semantic features to be transmitted are eavesdropped due to the leakage of the semantic model parameters, and the semantic communication security is further ensured by the data obfuscation of the row dimension, column dimension and channel dimension. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0049] Figure 1It is a flow chart of a semantic communication method for a power Internet of Things terminal provided by an embodiment of the present invention;

[0050] Figure 2 This is an architecture diagram of a semantic communication system for power Internet of Things terminals provided by an embodiment of the present invention;

[0051] Figure 3 is a schematic diagram of the structure of a semantic codec based on Swin Transformer provided in an embodiment of the present invention;

[0052] Figure 4 is a schematic diagram of the structure of a Swin Transformer module provided in an embodiment of the present invention;

[0053] Figure 5 It is an implementation architecture diagram of a semantic communication method based on data obfuscation provided by an embodiment of the present invention;

[0054] Figure 6 is a schematic diagram of a process of row obfuscation provided by an embodiment of the present invention;

[0055] Figure 7 is a schematic diagram of a column obfuscation process provided by an embodiment of the present invention;

[0056] Figure 8 is a schematic diagram of a channel obfuscation process provided by an embodiment of the present invention;

[0057] Fig. 9 It is a schematic diagram of image restoration quality comparison between an eavesdropper and a power Internet of Things terminal when the receiving signal-to-noise ratio is 13 dB provided by an embodiment of the present invention;

[0058] Fig.10 It is a schematic diagram of comparing the LPIPS values ​​of an eavesdropper and a power Internet of Things terminal under different signal-to-noise ratios provided by an embodiment of the present invention;

[0059] Fig.11 It is a structural schematic diagram of a semantic communication device for power Internet of Things terminals provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0060] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present invention. However, it should be clear to those skilled in the art that the present invention may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present invention.

[0061] In order to make the purpose, technical solutions and advantages of the present invention more clear, specific embodiments will be described below in conjunction with the accompanying drawings.

[0062] Figure 1 The flowchart of the implementation of the semantic communication method for the power Internet of Things terminal provided by the embodiment of the present invention is as follows:

[0063] Step 101: receiving semantic model parameters of a target semantic model trained by a cloud platform.

[0064] like Figure 2 As shown, in this embodiment, in order to realize the application of semantic models in power Internet of Things terminals, a semantic communication system for power Internet of Things terminals is established, and the semantic communication system includes power Internet of Things terminal A, cloud platform and power Internet of Things terminal B. Among them, power Internet of Things terminal A and power Internet of Things terminal B can both be used as sending power Internet of Things terminals or target power Internet of Things terminals, and the cloud platform can be a Machine Learning as a Service (MLaaS) platform.

[0065] Among them, the cloud platform is used to receive the training data sent by the power Internet of Things terminal or the target power Internet of Things terminal; input the image in the training data as the original image into the semantic encoder based on Swin Transformer, and obtain the corresponding reconstructed image output by the semantic decoder based on Swin Transformer; calculate the mean square error between each reconstructed image and the corresponding original image, and obtain the loss function according to the mean square error; based on the Adam optimizer, training is performed with the minimum loss function as the objective function to obtain the target semantic model and the semantic model parameters of the target semantic model.

[0066] The power Internet of Things terminal is sent to receive the semantic model parameters of the target semantic model trained by the cloud platform; the local semantic model is updated based on the semantic model parameters, and the image data to be transmitted is encoded according to the updated local semantic model to obtain the semantic features to be transmitted; the local semantic model and the target semantic model have the same structure; the row dimension, column dimension and channel dimension of the semantic features to be transmitted are sequentially obfuscated, the data obfuscation scheme is recorded and the obfuscated semantic features are obtained; the data obfuscation scheme and the obfuscated semantic features are encrypted and transmitted to the target power Internet of Things terminal.

[0067] The target power Internet of Things terminal is used to receive the semantic model parameters of the target semantic model trained by the cloud platform, the encrypted data obfuscation scheme and the encrypted obfuscated semantic features; decrypt the encrypted data obfuscation scheme and the encrypted obfuscated semantic features to obtain the data obfuscation scheme and the obfuscated semantic features; deobfuscate the obfuscated semantic features based on the data obfuscation scheme to obtain the semantic features to be transmitted; update the local semantic model based on the semantic model parameters, and decode the semantic features to be transmitted according to the updated local semantic model to obtain a reconstructed image of the image to be transmitted.

[0068] Exemplarily, taking the power Internet of Things terminal A as a sending power Internet of Things terminal as an example, the semantic communication method for the power Internet of Things terminal provided in this embodiment is described:

[0069] First, the resource-constrained power IoT terminal A can upload the training data to the MLaaS platform. Then, the MLaaS platform designs the semantic codec based on deep learning, simulates the semantic transmission process of the data, and completes the semantic model training on the MLaaS platform. After the training is completed, the MLaaS platform transmits the semantic model parameters back to the power IoT terminal A and its semantic data transmission object, such as the power IoT terminal B. In addition, by placing the computationally complex semantic model training process on the MLaaS platform, the consumption of local computing resources is reduced.

[0070] Exemplarily, for a cloud platform such as an MLaaS platform, the process of obtaining the semantic model parameters of the target semantic model may include:

[0071] Receive training data sent by the power Internet of Things terminal.

[0072] The images in the training data are input as original images into the Swin Transformer-based semantic encoder, and the corresponding reconstructed images output by the Swin Transformer-based semantic decoder are obtained.

[0073] The mean square error between each reconstructed image and the corresponding original image is calculated, and the loss function is obtained based on the mean square error.

[0074] Based on the Adam optimizer, training is performed with the minimum loss function as the objective function to obtain the semantic model parameters of the target semantic model.

[0075] Exemplarily, the loss function may be:

[0076]

[0077] in, is the loss function, i = 1, 2, ..., N, N is the number of images in the training data, s iis the i-th original image, is the i-th reconstructed image, is the mean square error between the i-th reconstructed image and the i-th original image.

[0078] Exemplarily, the objective function may be:

[0079]

[0080] Among them, α and β represent the weight and bias parameters of the neural network that constitute the target semantic model, respectively. is the original image s and the reconstructed image The joint probability distribution of Represents the loss function In the joint probability distribution Seek hope.

[0081] Combination Figure 3 As shown, in this embodiment, the MLaaS platform can use a semantic codec based on Swin Transformer as a semantic model for training. The channel between the semantic encoder and the semantic decoder can use Joint Source Channel Coding (JSCC). JSCC is a coding method that takes into account both source coding and channel coding. Source coding is used to compress data, and channel coding is used to protect data from noise during transmission, thereby improving the overall performance of semantic coding.

[0082] The Swin Transformer-based semantic encoder may include a Patch Embedding layer, N1 Swin Transformer modules, a Patch Merging layer, and N2 Swin Transformer modules. The Swin Transformer-based semantic decoder is a symmetric structure of the Swin Transformer-based semantic encoder, and may include N1 Swin Transformer modules, a Patch Division layer, N2 Swin Transformer modules, and a Patch Division layer.

[0083] During the training process, the images in the training data are used as the original images (Where H and W represent the height and width of the original image, respectively, 3 represents the number of channels, and an image generally has three channels: R, G, and B). The image features are obtained through the Patch Embedding layer. Where C1 represents the number of channels of the original image after compression.

[0084] After that, the image features enter the Swin Transformer module, which is composed as follows Figure 4 As shown in Figure 1, when the image feature initially enters the Swin Transformer module, l-1=0, where l represents the image feature entering the Swin Transformer module for the lth time. The Swin Transformer module first uses layer normalization (LN) to transform z l-1 Normalization to obtain LN(z l-1 ), and then the Windows Multihead Self-Attention (WMSA) module is used to calculate self-attention only for the elements within the window, which improves the computational efficiency and enhances the model's ability to capture local features and global context. l-1 Add to get features

[0085]

[0086] Through LN and Multi-Layer Perceptron (MLP) processing, z is obtained l :

[0087]

[0088] z l Processed by LN and Shifted Window Multihead SelfAttention (SWMSA) and combined with feature z l Add together to get

[0089]

[0090] Thus, by shifting the window, image information of different ranges can be efficiently integrated to enhance the model’s understanding ability. Finally, it is processed by LN and MLP modules and combined with the feature Add together and finally get feature z l+1 :

[0091]

[0092] This completes a complete Swin Transformer module operation.

[0093] After being processed by N1 Swin Transformer modules, Patch Merging is performed to compress the image to Where C2 represents the number of channels of the image after compression.

[0094] Then, after processing through N2 Swin Transformer modules, we get Next, let Through the normalization layer to constrain the transmit power, the channel input z is obtained:

[0095]

[0096] in yes is the conjugate transpose of , P is the average transmit power, and k is the length vector that maps the n-dimensional input image (i.e., the original image) to the complex channel input sample z.

[0097] The MLaaS platform receives the original image s, It is mapped into complex-valued channel input symbols through a semantic encoder If the original image dimension n is denoted as the source bandwidth and the channel dimension k is denoted as the channel bandwidth, then k / n is called the bandwidth compression ratio.

[0098] Channel coding uses JSCC. Assuming that the wireless channel is an additive white Gaussian noise (AWGN) channel and is modeled as a non-trainable layer, the channel output can be expressed as in is the AWGN vector.

[0099] The semantic decoder is a symmetric architecture of the semantic encoder. The operations performed by the encoder are then flipped through the Swin Transformer module and Patch Division to map the image features into estimates of the original image. That is, reconstruct the image.

[0100] make As a loss function to evaluate the reconstructed image The loss between the original image s and the reconstructed image is obtained by using the Adam optimizer to update the model parameters. The average distortion between is minimized, thereby obtaining the training parameters of the semantic codec model, that is, the semantic model parameters of the target semantic model:

[0101] After the target semantic model and semantic model parameters are obtained through training on the MLaaS platform, the semantic model parameters are transmitted back to the power IoT terminal A and the power IoT terminal B. Then, the power IoT terminal A and the power IoT terminal B use the trained semantic encoder and semantic decoder locally to realize the semantic wireless transmission between the power IoT terminal A and the power IoT terminal B. In this way, the computing tasks of the semantic training phase based on JSCC and the storage of the semantic knowledge base are executed on the MLaaS platform, thereby reducing the local computing resource consumption of the power IoT terminal and realizing the application of semantic communication in the power IoT terminal.

[0102] Moreover, the characteristic of semantic communication is that it transmits the meaning of parsed information rather than the original data itself. Compared with traditional data transmission methods, this method increases the ambiguity of information to a certain extent, making it more difficult for unauthorized third parties to interpret information, which naturally forms an implicit security protection. Therefore, for individuals who want to eavesdrop, unless they know for sure that the target is using semantic communication and successfully obtains the relevant network model and parameters, it is difficult to achieve effective eavesdropping, thereby improving the communication security between power Internet of Things terminals to a certain extent.

[0103] Step 102, updating the local semantic model based on the semantic model parameters, and encoding the image data to be transmitted according to the updated local semantic model to obtain the semantic features to be transmitted; the local semantic model and the target semantic model have the same structure.

[0104] Step 103, sequentially perform data obfuscation on the row dimension, column dimension, and channel dimension of the semantic feature to be transmitted, record the data obfuscation scheme, and obtain the obfuscated semantic feature.

[0105] Step 104: encrypt the data obfuscation scheme and obfuscation semantic features and transmit them to the target power Internet of Things terminal.

[0106] This embodiment considers an extreme case, that is, the MLaaS platform is a malicious cloud provider, and the eavesdropper can obtain the complete semantic model parameters from it. During the local semantic transmission process of the legitimate device, the eavesdropper can use the obtained decoder and parameters to obtain the received semantic signal. Reconstructed image Right now:

[0107]

[0108] in represents the decoder illegally obtained by the eavesdropper, θ e Represents the eavesdropper's semantic decoder parameters.

[0109] Therefore, an anti-eavesdropping method based on data obfuscation is further designed to solve the communication problem of the MLaaS platform in the semantic reasoning stage (i.e., the semantic communication stage). Figure 5 As shown, after the power Internet of Things terminal A uses the trained semantic encoder locally to implement semantic encoding, the encoded semantic features to be transmitted are randomly arranged in row dimensions, column dimensions and channel dimensions through the data obfuscation module to obtain obfuscated semantic features and data obfuscation schemes, and then the obfuscated semantic features and data obfuscation schemes are encrypted and transmitted to the power Internet of Things terminal B (that is, the target power Internet of Things terminal). The power Internet of Things terminal B decrypts the received obfuscated data (that is, the obfuscated semantic features and data obfuscation schemes) through the shared key and restores the original semantic data, thereby further ensuring the security of semantic communication between power Internet of Things terminals and realizing data privacy protection.

[0110] In one embodiment, the data obfuscation scheme includes a row data obfuscation scheme, a column data obfuscation scheme, and a channel data obfuscation scheme.

[0111] The row dimension, column dimension and channel dimension of the semantic feature to be transmitted are sequentially obfuscated, the data obfuscation scheme is recorded and the obfuscated semantic feature is obtained, which may include:

[0112] The row dimension of the semantic features to be transmitted is randomly arranged to obtain row confusion semantic features, and the corresponding random arrangement results are recorded as the row data confusion scheme.

[0113] The column dimension of the row confusion semantic feature is randomly arranged to obtain the column confusion semantic feature, and the corresponding random arrangement result is recorded as the column data confusion scheme.

[0114] The channel dimension of the column confusion semantic feature is randomly arranged to obtain the confusion semantic feature, and the corresponding random arrangement result is recorded as the channel data confusion scheme.

[0115] Exemplarily, randomly arranging the row dimension of the semantic feature to be transmitted to obtain the row obfuscated semantic feature, and recording the corresponding random arrangement result as the row data obfuscation scheme may include:

[0116] An array consisting of row index values ​​corresponding to each row of features in the semantic features to be transmitted is obtained, which is recorded as the original row index array.

[0117] Each row index value in the original row index array is randomly arranged, and the random arrangement result of the original row index array is recorded as a row data obfuscation scheme.

[0118] The order of each row feature in the semantic feature to be transmitted is adjusted according to the order of each row index value in the row data obfuscation scheme to obtain the row obfuscated semantic feature.

[0119] Exemplarily, randomly arranging the column dimension of the row obfuscation semantic feature to obtain the column obfuscation semantic feature, and recording the corresponding random arrangement result as the column data obfuscation scheme may include:

[0120] Get an array consisting of column index values ​​corresponding to each column feature in the row confusion semantic feature, and record it as the original column index array.

[0121] The column index values ​​in the original column index array are randomly arranged, and the random arrangement result of the original column index array is recorded as a column data obfuscation scheme.

[0122] The order of each column feature in the row confusion semantic feature is adjusted according to the order of each column index value in the column data confusion scheme to obtain the column confusion semantic feature.

[0123] Exemplarily, randomly arranging the channel dimensions of the column obfuscation semantic features to obtain obfuscated semantic features, and recording the corresponding random arrangement results as the channel data obfuscation scheme may include:

[0124] Get an array consisting of channel index values ​​corresponding to each channel feature in the column confusion semantic feature, and record it as the original channel index array.

[0125] The channel index values ​​in the original channel index array are randomly arranged, and the random arrangement result of the original channel index array is recorded as a channel data obfuscation scheme.

[0126] The order of each channel feature in the column confusion semantic feature is adjusted according to the order of each channel index value in the channel data confusion scheme to obtain the confusion semantic feature.

[0127] Combination Figure 2 and Figure 5 As shown, this embodiment considers the MLaaS platform that completes the semantic model training and transmits the semantic encoding and decoding model parameters back to the power Internet of Things terminal. At the same time, the eavesdropper can also obtain the semantic model parameters from the malicious MLaaS platform through an insecure API interface, network monitoring or internal transactions.

[0128] Therefore, in the semantic reasoning stage, the power Internet of Things terminal A transmits the image data s to the power Internet of Things terminal B, and generates the semantic features y to be transmitted through the local encoder module (i.e., the local semantic model):

[0129]

[0130] in, represents the local semantic encoder output, θ a Represents the local semantic encoder parameters, which are updated based on the semantic model parameters sent back by the cloud platform.

[0131] Then, let y pass through the data obfuscation module, which randomly arranges the semantic features to be transmitted.

[0132] Exemplarily, the data obfuscation process performed by the data obfuscation module may include:

[0133] 1. Row confusion: Figure 6 As shown, for the semantic features to be transmitted It can be regarded as a tensor. First, the tensor can be randomly arranged along the row dimension. That is, the array [y0, y1, ..., y h-1 ] is defined as u, where each element of [0,1,...,h-1] represents the row index value corresponding to each row feature in the semantic feature y to be transmitted. After applying u, the semantic feature to be transmitted after random permutation of rows is obtained.

[0134] 2. Column confusion: Figure 7 As shown, define v as an array Any random permutation of , where each element of [0,1,...,w-1] represents the semantic features to be transmitted after the rows are randomly permuted The column index value corresponding to each column feature in . After applying v, the semantic features to be transmitted after the columns are randomly arranged are obtained

[0135] 3. Channel confusion: Figure 8 As shown, define q as an array Any random permutation of , where each element of [0,1,...,c-1] represents the semantic features to be transmitted after the columns are randomly permuted The channel index value corresponding to each channel feature in . After applying q, the confused semantic feature is obtained:

[0136]

[0137] in, represents the output after data obfuscation, Represents the data obfuscation scheme, that is, the corresponding row, column, and channel arrangement schemes u, v, q.

[0138] Then use the shared key between the power Internet of Things terminal A and the power Internet of Things terminal B to encrypt the encrypted permutation scheme u, v, q and Transmitted together to the power Internet of Things terminal B.

[0139] The power Internet of Things terminal B receives the arrangement scheme u, v, q from the transmitter (that is, the power Internet of Things terminal A) and First, use the shared key to decrypt the permutation scheme u, v, q, and then use the decoded u, v, q to The obfuscated data is restored and finally obtained through the semantic decoder:

[0140]

[0141] in, represents the output of the decryptor, Represents the decryptor parameters, represents the semantic decoder output of power IoT terminal B, θ b represents the semantic decoder parameters, Represents the reconstructed image output by the power Internet of Things terminal B.

[0142] The eavesdropper obtains the trained semantic model parameters through the malicious MLaaS platform, and then eavesdrops on the image data s when the power Internet of Things terminal A communicates with the power Internet of Things terminal B. Since the eavesdropper does not obtain the relevant content of feature confusion from the MLaaS platform, it directly uses the semantic decoder and its parameters obtained from the MLaaS platform to try to obtain the received feature data. Restore the image and get:

[0143]

[0144] in, Represents the image recovered by the eavesdropper.

[0145] Since the power IoT terminal A and the power IoT terminal B synchronize the data obfuscation scheme through a shared key, which is used to encrypt and decrypt the data obfuscation arrangement, even if the eavesdropper obtains the model parameters of the MLaaS platform, it cannot effectively restore the obfuscated semantic data.

[0146] In this embodiment, considering that due to the broadcast characteristics of the wireless channel, the eavesdropper can receive the semantic information sent by the power Internet of Things terminal A. If it can also obtain the semantic codec parameters through the MLaaS platform, it will be able to eavesdrop on A's data. In order to prevent the eavesdropper from eavesdropping by obtaining the semantic model parameters, this embodiment introduces a data obfuscation module. Then, in the semantic reasoning stage, the power Internet of Things terminal A randomly arranges the rows, columns and channels of the generated semantic features to be transmitted (i.e., data obfuscation), and encrypts the obfuscated information with a shared key and transmits it to the power Internet of Things terminal B. After receiving the obfuscated semantic data and the data obfuscation scheme, the power Internet of Things terminal B uses the shared key to decrypt the data obfuscation scheme, and restores the transmitted original image or other semantic information through the semantic decoder. Since the data is obfuscated during the transmission process, even if the eavesdropper obtains the semantic model parameters, it cannot reconstruct the original semantic information, thereby further ensuring the semantic communication security of the power Internet of Things terminal.

[0147] In order to prove the effectiveness of the semantic communication method for power Internet of Things terminals provided in this embodiment when facing passive eavesdroppers, an experimental verification is carried out for the image data transmission task:

[0148] During the verification process, the neural network (i.e., semantic codec) was trained using the CIFAR10 image dataset. The training data includes 50,000 RGB images with a resolution of 32×32. Then, an evaluation was performed on 10,000 test images in the CIFAR10 dataset to compare the eavesdropper's eavesdropping performance and the effectiveness of the semantic communication method for power Internet of Things terminals provided in this embodiment. In order to measure the image quality, the three indicators of Peak Signal-to-Noise Ratio (PSNR), Multi-Scale Structural Similarity Index (MS-SSIM) and Learned Perceptual Image Patch Similarity (LPIPS) were used for evaluation, where the higher the values ​​of PSNR and MS-SSIM, the better the image quality, and the lower the value of LPIPS, the more similar the two images are, and vice versa. Assume that both the communication channel and the eavesdropping channel are AWGN channels, the signal-to-noise ratio is set to 1dB, 4dB, 7dB, 10dB and 13dB respectively, the bandwidth compression ratio is set to 1 / 16, N1 and N2 are 2 and 4 respectively, C1 and C2 are 128 and 256 respectively. Adam is used as the optimizer, and the initial learning rate is set to 10 -4 .

[0149] like Fig. 9 As shown, by comparing the eavesdropper's eavesdropping image quality and the image quality restored by the power Internet of Things terminal when the receiving signal-to-noise ratio is 13dB, it can be observed that the eavesdropped image is visually unrecognizable, while the power Internet of Things terminal can perfectly restore the original image, which proves the effectiveness of the semantic communication method for the power Internet of Things terminal proposed in this embodiment in preventing the eavesdropper from eavesdropping on the original image.

[0150] like Fig.10As shown, the LPIPS values ​​of the eavesdropper and the power Internet of Things terminal are compared and analyzed. LPIPS relies on a pre-trained deep learning model, and evaluates similarity by extracting and comparing the distance of image features. This method is closer to human perception. The data shows that the LPIPS value of the power Internet of Things terminal is below 0.1, and the value even drops to 0.005 when the signal-to-noise ratio reaches 13dB, which is almost the same as the original image. In contrast, the LPIPS value of the eavesdropper generally exceeds 0.5, which means that the image information they can recover is very limited. Even under the condition of 13dB signal-to-noise ratio where they perform best, it is difficult to identify the original image from the received image. This result fully demonstrates that the semantic communication method for the power Internet of Things terminal proposed in this embodiment is extremely effective in preventing eavesdroppers from obtaining meaningful information in the image.

[0151] It should be noted that this embodiment introduces a semantic communication system built on the MLaaS platform, and at the same time pays attention to the potential eavesdropping risk of the system. In order to ensure that the designed system can accurately complete the semantic transmission task while maintaining the universality and information security of the power Internet of Things terminal, data obfuscation technology is used for encryption processing to prevent potential eavesdroppers from intercepting and understanding the semantic content in transmission. Through a series of simulation experiments, it is confirmed that the introduction of data obfuscation strategy can effectively enhance the performance of the semantic communication system in privacy protection.

[0152] The embodiment of the present invention receives the semantic model parameters of the target semantic model obtained by training on the cloud platform, and updates the local semantic model based on the semantic model parameters, so as to complete the training of the target semantic model on the cloud platform and transmit the model parameters back to the power Internet of Things terminal, thereby reducing the energy consumption and time consumption of the power Internet of Things terminal. On this basis, the image data to be transmitted is encoded according to the updated local semantic model to obtain the semantic features to be transmitted, and then the row dimension, column dimension and channel dimension of the semantic features to be transmitted are sequentially obfuscated, the data obfuscation scheme is recorded and the obfuscated semantic features are obtained, and finally the data obfuscation scheme and the obfuscated semantic features are encrypted and transmitted to the target power Internet of Things terminal, thereby avoiding the situation where the semantic features to be transmitted are eavesdropped due to the leakage of the semantic model parameters, and the semantic communication security is further ensured by the data obfuscation of the row dimension, column dimension and channel dimension.

[0153] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0154] The following is an embodiment of the device of the present invention. For details not described in detail therein, reference may be made to the corresponding method embodiment described above.

[0155] Fig.11 The following is a schematic diagram of the structure of a semantic communication device for a power Internet of Things terminal provided by an embodiment of the present invention. For ease of explanation, only the parts related to the embodiment of the present invention are shown, which are described in detail as follows:

[0156] like Fig.11 As shown, the semantic communication device for the power Internet of Things terminal includes: a receiving module 111, a semantic encoding module 112, a data obfuscation module 113 and a transmission module 114.

[0157] A receiving module 111 is used to receive semantic model parameters of a target semantic model trained by a cloud platform;

[0158] A semantic encoding module 112, used for updating the local semantic model based on the semantic model parameters, and encoding the image data to be transmitted according to the updated local semantic model to obtain the semantic features to be transmitted; the local semantic model and the target semantic model have the same structure;

[0159] A data obfuscation module 113 is used to sequentially perform data obfuscation on the row dimension, column dimension and channel dimension of the semantic feature to be transmitted, record the data obfuscation scheme and obtain the obfuscated semantic feature;

[0160] The transmission module 114 is used to encrypt the data obfuscation scheme and obfuscation semantic features and transmit them to the target power Internet of Things terminal.

[0161] The embodiment of the present invention receives the semantic model parameters of the target semantic model obtained by training on the cloud platform, and updates the local semantic model based on the semantic model parameters, so as to complete the training of the target semantic model on the cloud platform and transmit the model parameters back to the power Internet of Things terminal, thereby reducing the energy consumption and time consumption of the power Internet of Things terminal. On this basis, the image data to be transmitted is encoded according to the updated local semantic model to obtain the semantic features to be transmitted, and then the row dimension, column dimension and channel dimension of the semantic features to be transmitted are sequentially obfuscated, the data obfuscation scheme is recorded and the obfuscated semantic features are obtained, and finally the data obfuscation scheme and the obfuscated semantic features are encrypted and transmitted to the target power Internet of Things terminal, thereby avoiding the situation where the semantic features to be transmitted are eavesdropped due to the leakage of the semantic model parameters, and the semantic communication security is further ensured by the data obfuscation of the row dimension, column dimension and channel dimension.

[0162] In one possible implementation, the data obfuscation scheme includes a row data obfuscation scheme, a column data obfuscation scheme and a channel data obfuscation scheme; the data obfuscation module 113 can be used to randomly arrange the row dimension of the semantic feature to be transmitted to obtain the row obfuscated semantic feature, and record the corresponding random arrangement result as the row data obfuscation scheme; randomly arrange the column dimension of the row obfuscated semantic feature to obtain the column obfuscated semantic feature, and record the corresponding random arrangement result as the column data obfuscation scheme; randomly arrange the channel dimension of the column obfuscated semantic feature to obtain the obfuscated semantic feature, and record the corresponding random arrangement result as the channel data obfuscation scheme.

[0163] In one possible implementation, the data obfuscation module 113 can be used to obtain an array consisting of row index values ​​corresponding to each row feature in the semantic feature to be transmitted, recorded as the original row index array; randomly arrange the row index values ​​in the original row index array, and record the random arrangement result of the original row index array as a row data obfuscation scheme; adjust the order of each row feature in the semantic feature to be transmitted according to the order of each row index value in the row data obfuscation scheme to obtain a row obfuscated semantic feature.

[0164] In one possible implementation, the data obfuscation module 113 can be used to obtain an array consisting of column index values ​​corresponding to each column feature in the row obfuscation semantic feature, recorded as the original column index array; randomly arrange the column index values ​​in the original column index array, and record the random arrangement result of the original column index array as the column data obfuscation scheme; adjust the order of each column feature in the row obfuscation semantic feature according to the order of each column index value in the column data obfuscation scheme to obtain the column obfuscation semantic feature.

[0165] In one possible implementation, the data obfuscation module 113 can be used to obtain an array consisting of channel index values ​​corresponding to each channel feature in the column obfuscation semantic feature, recorded as the original channel index array; randomly arrange the channel index values ​​in the original channel index array, and record the random arrangement result of the original channel index array as the channel data obfuscation scheme; adjust the order of each channel feature in the column obfuscation semantic feature according to the order of each channel index value in the channel data obfuscation scheme to obtain the obfuscated semantic feature.

[0166] In a possible implementation, the process of obtaining the semantic model parameters of the target semantic model includes: receiving training data sent by the power Internet of Things terminal; inputting the image in the training data as the original image into the semantic encoder based on Swin Transformer, and obtaining the corresponding reconstructed image output by the semantic decoder based on Swin Transformer; calculating the mean square error between each reconstructed image and the corresponding original image, and obtaining the loss function according to the mean square error; based on the Adam optimizer, training is performed with the minimum loss function as the objective function to obtain the semantic model parameters of the target semantic model.

[0167] In one possible implementation, the loss function is:

[0168]

[0169] in, is the loss function, i = 1, 2, ..., N, N is the number of images in the training data, s i is the i-th original image, is the i-th reconstructed image, is the mean square error between the i-th reconstructed image and the i-th original image.

[0170] In one possible implementation, the objective function is:

[0171]

[0172] Among them, α and β represent the weight and bias parameters of the neural network that constitute the target semantic model, respectively. is the original image s and the reconstructed image The joint probability distribution of Represents the loss function In the joint probability distribution Seek hope.

[0173] In one possible implementation, the cloud platform is a machine learning as a service platform.

[0174] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0175] Those of ordinary skill in the art will appreciate that the templates, units, and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0176] If the module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, it can implement the steps of the above-mentioned semantic communication method embodiments for power Internet of Things terminals. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. Computer-readable media may include: any entity or device that can carry computer program code, recording media, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory, random access memory, electrical carrier signal, telecommunication signal and software distribution medium, etc.

[0177] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention.

Claims

1. A semantic communication method for power Internet of Things terminals, characterized in that: include: Receive semantic model parameters of the target semantic model trained by the cloud platform; The local semantic model is updated based on the semantic model parameters, and the image data to be transmitted is encoded according to the updated local semantic model to obtain the semantic features to be transmitted; the local semantic model and the target semantic model have the same structure; Sequentially performing data obfuscation on the row dimension, column dimension, and channel dimension of the semantic feature to be transmitted, recording the data obfuscation scheme and obtaining the obfuscated semantic feature; The data obfuscation scheme and the obfuscated semantic features are encrypted and transmitted to the target power Internet of Things terminal.

2. The semantic communication method for power Internet of Things terminals according to claim 1 is characterized in that: The data obfuscation scheme includes a row data obfuscation scheme, a column data obfuscation scheme and a channel data obfuscation scheme; Sequentially performing data obfuscation on the row dimension, column dimension, and channel dimension of the semantic feature to be transmitted, recording the data obfuscation scheme and obtaining the obfuscated semantic feature, including: Randomly arrange the row dimensions of the semantic features to be transmitted to obtain row obfuscated semantic features, and record the corresponding random arrangement results as a row data obfuscation scheme; Randomly arrange the column dimensions of the row confusion semantic features to obtain column confusion semantic features, and record the corresponding random arrangement results as a column data confusion scheme; The channel dimensions of the column obfuscated semantic features are randomly arranged to obtain obfuscated semantic features, and the corresponding random arrangement results are recorded as a channel data obfuscation scheme.

3. The semantic communication method for power Internet of Things terminals according to claim 2 is characterized in that: Randomly arranging the row dimensions of the semantic features to be transmitted to obtain row obfuscated semantic features, and recording the corresponding random arrangement results as a row data obfuscation scheme, including: Obtaining an array consisting of row index values ​​corresponding to each row of features in the semantic features to be transmitted, recorded as an original row index array; Randomly arrange each row index value in the original row index array, and record the random arrangement result of the original row index array as a row data obfuscation scheme; The order of each row feature in the semantic features to be transmitted is adjusted according to the order of each row index value in the row data obfuscation scheme to obtain a row obfuscated semantic feature.

4. The semantic communication method for power Internet of Things terminals according to claim 2 is characterized in that: The column dimension of the row confusion semantic feature is randomly arranged to obtain a column confusion semantic feature, and the corresponding random arrangement result is recorded as a column data obfuscation scheme, including: Obtaining an array consisting of column index values ​​corresponding to each column feature in the row confusion semantic feature, recorded as an original column index array; Randomly arrange the column index values ​​in the original column index array, and record the random arrangement result of the original column index array as a column data obfuscation scheme; The order of each column feature in the row confusion semantic feature is adjusted according to the order of each column index value in the column data obfuscation scheme to obtain a column confusion semantic feature.

5. The semantic communication method for power Internet of Things terminals according to claim 2 is characterized in that: The channel dimensions of the column obfuscated semantic features are randomly arranged to obtain obfuscated semantic features, and the corresponding random arrangement results are recorded as a channel data obfuscation scheme, including: Obtain an array consisting of channel index values ​​corresponding to each channel feature in the column confusion semantic feature, recorded as an original channel index array; Randomly arrange each channel index value in the original channel index array, and record the random arrangement result of the original channel index array as a channel data obfuscation scheme; The order of each channel feature in the column obfuscation semantic feature is adjusted according to the order of each channel index value in the channel data obfuscation scheme to obtain the obfuscated semantic feature.

6. The semantic communication method for power Internet of Things terminals according to claim 1 is characterized in that: The process of obtaining the semantic model parameters of the target semantic model includes: Receive training data sent by the power Internet of Things terminal; Input the image in the training data as the original image into the semantic encoder based on Swin Transformer, and obtain the reconstructed image output by the corresponding semantic decoder based on Swin Transformer; Calculating a mean square error between each of the reconstructed images and the corresponding original image, and obtaining a loss function according to the mean square error; Based on the Adam optimizer, training is performed with the minimum loss function as the objective function to obtain the semantic model parameters of the target semantic model.

7. The semantic communication method for power Internet of Things terminals according to claim 6 is characterized in that: The loss function is: in, is the loss function, i=1,2,…,N, N is the number of images in the training data, s i is the i-th original image, is the i-th reconstructed image, is the mean square error between the i-th reconstructed image and the i-th original image.

8. The semantic communication method for power Internet of Things terminals according to claim 6 is characterized in that: The objective function is: Wherein, α and β represent the weight and bias parameters of the neural network constituting the target semantic model, respectively. is the original image s and the reconstructed image The joint probability distribution of Represents the loss function In the joint probability distribution Seek hope.

9. A semantic communication device for power Internet of Things terminals, characterized in that: include: A receiving module, used for receiving semantic model parameters of a target semantic model trained by a cloud platform; A semantic encoding module, used for updating the local semantic model based on the semantic model parameters, and encoding the image data to be transmitted according to the updated local semantic model to obtain semantic features to be transmitted; the local semantic model and the target semantic model have the same structure; A data obfuscation module, used to sequentially perform data obfuscation on the row dimension, column dimension and channel dimension of the semantic feature to be transmitted, record the data obfuscation scheme and obtain the obfuscated semantic feature; The transmission module is used to encrypt the data obfuscation scheme and the obfuscated semantic features and transmit them to the target power Internet of Things terminal.

10. A semantic communication system for power Internet of Things terminals, characterized in that: include: Cloud platform, sending power IoT terminal and target power IoT terminal; The cloud platform is used to receive the training data sent by the sending power Internet of Things terminal or the target power Internet of Things terminal; Input the images in the training data as original images into a semantic encoder based on Swin Transformer, and obtain the reconstructed images output by the corresponding semantic decoder based on Swin Transformer; calculate the mean square error between each of the reconstructed images and the corresponding original image, and obtain a loss function according to the mean square error; based on the Adam optimizer, train with the minimum loss function as the objective function to obtain a target semantic model and semantic model parameters of the target semantic model; The sending power Internet of Things terminal is used to receive the semantic model parameters of the target semantic model trained by the cloud platform; update the local semantic model based on the semantic model parameters, and encode the image data to be transmitted according to the updated local semantic model to obtain the semantic features to be transmitted; the local semantic model and the target semantic model have the same structure; sequentially perform data obfuscation on the row dimension, column dimension and channel dimension of the semantic features to be transmitted, record the data obfuscation scheme and obtain the obfuscated semantic features; encrypt the data obfuscation scheme and the obfuscated semantic features and transmit them to the target power Internet of Things terminal; The target power Internet of Things terminal is used to receive the semantic model parameters of the target semantic model trained by the cloud platform, the encrypted data obfuscation scheme and the encrypted obfuscated semantic features; Decrypting the encrypted data obfuscation scheme and the encrypted obfuscated semantic features to obtain the data obfuscation scheme and the obfuscated semantic features; deobfuscating the obfuscated semantic features based on the data obfuscation scheme to obtain the semantic features to be transmitted; The local semantic model is updated based on the semantic model parameters, and the semantic features to be transmitted are decoded according to the updated local semantic model to obtain a reconstructed image of the image to be transmitted.

Images