A method for constructing a semantic broadcast communication network based on feature decoupling

Abstract

The present invention provides a method for constructing a semantic broadcast communication network based on feature decoupling, wherein the transmitter includes a feature selection module, and each user has a feature completion module. The semantic encoder does not extract all features, but extracts decoupled semantic features, and then only selects the semantic features expected by the user for broadcasting, which can further improve the transmission efficiency; an optimal distortion distribution scheme for multi-source data compression is proposed, and the inner and outer limits of the reachable rate area of ​​the semantic broadcast channel are derived. In order to realize the proposed semantic broadcast framework, a lightweight robust semantic broadcast communication network is designed by utilizing a supervised autoencoder. The present invention also designs a hardware concept verification prototype of the semantic broadcast network for the first time, and the proposed semantic broadcast network can be implemented in real time. Experiments show that the robust semantic broadcast communication network proposed in the present invention can significantly improve the transmission efficiency.

Description

Technical Field

[0001] The present invention relates to the field of semantic communication, and in particular to a method for constructing a semantic broadcast communication network based on feature decoupling. Background Art

[0002] Due to the increasing quality of service requirements of various IoT devices, the fifth generation (5G) communication networks face huge challenges, such as massive data traffic, ultra-high speed and extremely low latency requirements driven by holographic communication and extremely high-definition video transmission applications. Cisco predicts that the number of devices supporting mobile Internet will reach 2.93 billion by the end of 2023. And the International Telecommunication Union estimates that wireless data traffic will reach 4394EB in 2030. Under the constraints of hardware cost and energy consumption, this explosive demand will gradually exceed the capacity of 5G networks. In order to expand 5G capabilities, semantic communication, which uses the computing power of transceivers to reduce the cost of transmission resources, has become a promising sixth generation (6G) communication technology. Compared with traditional bit-level communication systems, semantic communication only extracts and transmits task-related information, thereby significantly reducing the data traffic burden on communication networks.

[0003] Most existing studies have applied deep learning (DL) technology in transmission design, the main disadvantage of which is the unexplainable nature of the operation: the semantic features extracted from the source data are encoded and coupled together, which is unexplainable, i.e., a hidden representation method. This not only hinders the application of specific semantic features, but also causes undesired semantic features to be transmitted to the receiver, thereby reducing the transmission efficiency. In addition, current research mainly focuses on single-user point-to-point communication scenarios, while there is little research on multi-user semantic broadcast communication. In multi-user semantic broadcast communication, the expected semantic information of each user may be different, and the knowledge base of each user may also be different. Therefore, the single-user semantic communication scheme cannot be applied to multi-user semantic broadcast communication, and further in-depth research is needed. Summary of the invention

[0004] Purpose of the invention: The technical problem to be solved by the present invention is to provide a method for constructing a semantic broadcast communication network based on feature decoupling in view of the shortcomings of the prior art, including: establishing a network system framework module, an information theory metric calculation module, and a design module;

[0005] The network system framework module is used to extract semantic features and transmit semantic features;

[0006] The information theory metric calculation module is used to construct an optimal distortion allocation method for multi-source data semantic compression and establish a reachability region;

[0007] The design module is used to construct a lightweight and robust semantic broadcast communication network based on a supervised autoencoder to achieve controllable decoupling of semantic features.

[0008] The network system framework module includes a semantic source module, a semantic encoder module, a semantic decoder module, a feature selection module, a semantic broadcast channel module, and a feature completion module;

[0009] The semantic source module is used to generate original data. The generated original data X d Including expected semantic features and unexpected semantic features. The sender needs to send expected semantic features according to the interests of each user, but does not need to send unexpected semantic features.

[0010] The semantic encoder module is used to: based on a knowledge base (the knowledge base involves shared knowledge between the sender and the receiver), d Extract L decoupled semantic features from where z l represents the lth semantic feature, is a set of semantic feature numbers, namely The function of the semantic encoder is to convert high-dimensional data X d It is compressed into a low-dimensional semantic feature Z; on the other hand, channel coding is implemented and redundancy is included to resist channel changes.

[0011] The feature selection module is used to select semantic features for each user according to task requirements: represents the semantic features that user i is interested in, represents the set of serial numbers of the semantic features that user i is interested in, then Since user i only has If interested, the remaining features is regarded as the redundancy of user i; where z n represents redundant features, Redundant features are semantic features that are removed from the total semantic features that the user is interested in.

[0012] Let X i,s represents the selected semantic features of user i as follows:

[0013]

[0014] Then the selected semantic features X of K users s for:

[0015] The semantic broadcast channel module receives the signal Y from user i. i Decode and output the estimated expected features represents the lth estimated semantic feature; with input X s and output (Y 1,s ,…,Y K,s ) is represented as the conditional probability p(y 1,s ,…,y K,s |X s ), where y K,s Represents the signal variable received by the Kth user.

[0016] The feature completion module is used to: Not transmitted, user i automatically generates corresponding unexpected features by searching the knowledge base. Finally, by combining the expected features and unexpected features User i can obtain complete semantic features z l It is just a variable that represents semantic features. Whether it represents expected semantic features or unexpected semantic features depends on the value range of the subscript l. Here, if Then l indicates the expected semantic feature; otherwise, if Then l Indicates unexpected semantic features. represents the unexpected feature of user i, which is not transmitted. At the receiving end, the user automatically generates data by searching the knowledge base Used to fill in the data that has not been transmitted, so that the complete semantic features can be obtained in the end

[0017] Each user has a semantic decoder module based on the complete semantic features. The semantic decoder module of user i obtains the reconstructed data by utilizing the knowledge base Finally, the data will be reconstructed Send to a semantic destination.

[0018] The information theory metric calculation module converts the conditional probability p(y 1,s ,…,y K,s |X s ) is set to satisfy the following relationship:

[0019] Y i,s =G i,s X s +N i,s (2)

[0020] Among them, G i represents the effective channel gain of user i from the feature selection module to the feature completion module, N i,srepresents the semantic noise received by user i; set the received semantic noise N i,s It follows a non-Gaussian distribution with a variance of

[0021] The optimal distortion allocation method for multi-source data semantic compression is based on independent source data The optimal distortion distribution method, where L d is the number of independent information sources and L d >1. The goal of the method is to minimize the distortion allocation function R(D, P) of multi-source data compression under the constraints of square error distortion and perception. Specifically, the optimization formula of the distortion allocation function R(D, P) is:

[0022]

[0023] So that:

[0024]

[0025] Among them, the function min(·) means finding the minimum value; I(a; b) means the mutual information between a and b; E(·) means finding the expectation; Represents the source data X i,d and estimated values ​​of source data The distance between them; D represents the threshold of total distortion; d i (·) represents relative entropy KL distance (Kullback-Leibler distance); Representation data Distribution of Represents data X i,d Distribution of Represents an estimate of the source data X-based i,d The conditional probability distribution of ; represents the set of estimated values ​​of source data; P represents the threshold of relative entropy.

[0026] Consider multiple independent Gaussian distribution data in represents a mean of 0 and a variance of Gaussian distribution. Let D i Represents X i,d and The squared error distortion between is as follows:

[0027]

[0028] Among them, the parameter λ D and λ P is the solution to the following problem:

[0029]

[0030]

[0031] Due to the existence of non-Gaussian semantic channel noise, the classical Shannon capacity region result of the broadcast channel cannot be directly applied to the semantic broadcast channel. In view of this, the information theory metric calculation module is also used to establish the reachability region of the two-user degraded semantic broadcast communication: for two users, |G1|≥|G2|, σ 1,s =σ 2,s , where G1 and G2 represent the channel of user 1 and user 2 respectively, σ 1,s and σ 2,s They represent the noise standard deviation of user 1 and user 2 respectively.

[0032] The design module also performs the following steps: φ (·) represents the semantic encoder module, i.e., the feature extraction network, and the subscript φ represents the neural network parameter set of the semantic encoder module; first, at the sending end, E φ (·) Extract semantic features from the source data and decompose them into two or more independent and interpretable semantic features; secondly, through the semantic feature selection module, the sender broadcasts the semantic features to the expected user; finally, at the receiving end, when user i receives the expected semantic features, the knowledge base is used to complete the unexpected features and the semantic decoder module is used to decode the expected features. Decode the semantic features, where represents the semantic decoder of the i-th user, i=1,...,K, subscript θ i Represents the set of neural network parameters of the semantic decoder module of user i.

[0033] Specifically, in order to achieve controllable decoupled semantic feature learning, the design module is used to construct a lightweight and robust semantic broadcast communication network based on a supervised autoencoder (AE), which specifically includes the following steps:

[0034] Step 1, self-reconstruction training: semantic encoder module E φ (·) Use the following formula to convert the lth data Compressed into a latent semantic feature vector Z( l ):

[0035]

[0036] where l=1,...,L s , L s Represents the total amount of data; Indicates data Perform feature extraction;

[0037] Semantic feature vector Z (l) is broadcast to K semantic users; at the user end, the received semantic features for:

[0038]

[0039] where i = 1, ..., K; G i Represents the effective channel gain; N i,s represents the semantic noise received by user i;

[0040] User i decodes the received semantic features And get the reconstructed data as follows:

[0041]

[0042] in, Indicates the received semantic features Decode to obtain reconstructed data

[0043] Reconstruction loss function L based on standard supervised autoencoder AE,1 , for the semantic encoder module E φ (·) (with parameter set φ) and semantic decoder module (Parameter set is ) for joint optimization:

[0044]

[0045] Step 2: In order to improve the semantic feature extraction capability of the network, the lightweight robust semantic broadcast communication network is further trained by exchanging common semantic features and minimizing the reconstruction loss of common semantic feature exchange:

[0046] Select two input data with common semantic features and Extracted by semantic decoder module Semantic features of and Semantic features of set up The jth semantic feature of and The jth semantic feature of Same, exchange according to the following formula and The jth semantic feature of:

[0047]

[0048] in, represent Data after feature exchange; represent After feature exchange, j=1,...,L c , L c Represents the total number of common semantic features; Indicates that The jth semantic feature and The jth semantic feature of and is sent to K users;

[0049] At the receiving end, represents the semantic features received by user 1; Represents the semantic features received by user 2; using the semantic decoder module Reconstructed data and Reconstructed data as follows:

[0050]

[0051]

[0052] Reconstruction loss function L for common semantic feature exchange AE,2 as follows:

[0053]

[0054] Step 3: To enhance the decoupling capability of different semantic features, the lightweight robust semantic broadcast communication network is further trained by exchanging different semantic features and minimizing the reconstruction loss of exchanging different semantic features:

[0055] Select two input data with different semantic features and where l1,l3=1,...,L s , through the semantic encoder module E φ ,extract Semantic features of and Semantic features of

[0056]

[0057]

[0058] set up The nth semantic feature of and The nth semantic feature of Different, exchange according to the following formula and The nth semantic feature of:

[0059]

[0060] in, express Semantic features after feature exchange; express Semantic features after feature exchange, n=1,...,L e , L e Represents the total number of different features; after exchange and is transmitted to the user end.

[0061] On the receiving end, Receiving The semantic features of Receiving Semantic features:

[0062]

[0063]

[0064] Among them, α i is the equivalent channel gain, N i,s is the equivalent noise.

[0065] Through the semantic decoder module User i receives the semantic features The reconstructed data is The received semantic features The reconstructed data is

[0066]

[0067]

[0068] On this basis, the reconstructed data and As input data, repeat the semantic encoding in (14), the exchange of different semantic features in (15), the semantic feature propagation in (16) and the semantic feature decoding in (17).

[0069] After the two semantic features are exchanged, Represents input data The final reconstructed data, Represents input data The final reconstructed data, then the final reconstruction loss L AE,3 for:

[0070]

[0071] The proposed semantic broadcast communication network structure includes a transmitter and a user. Conv2D 64@64*64 means that the network consists of 64 2-D convolutional filters with an output size of 64×64. Dense 1×4096 means that a fully connected layer has 4096 neurons, and the residual block represents a convolutional block composed of two convolutional networks Conv2D 256@8*8 cascaded. d represents source data; Z represents semantic features; X s represents the semantic features after feature selection; Y i,s represents the semantic features estimated by user i; Represents the semantic features of user i after feature completion; represents the reconstructed data of user i.

[0072] The semantic broadcast network structure proposed in the present invention is as follows, where the symbol “→” represents cascade:

[0073] Semantic sending end network structure:

[0074] User-side network structure:

[0075] Beneficial effects: The invention proposes for the first time a practical semantic broadcast communication framework based on feature decoupling, which can utilize existing mature bit-level communication network standards and hardware. The proposed framework first extracts semantic information and decouples it into independent semantic features, and then through feature selection, only selects the semantic features of interest to the user for transmission without sending the remaining semantic features, which not only reduces the load of the broadcast communication network, but also increases the robustness of the semantic features to channel noise. Secondly, an optimal distortion allocation scheme for multi-source data compression is proposed, and the inner and outer bounds of the semantic broadcast channel in the reachability region are derived. Thirdly, the present invention designs a lightweight and robust semantic broadcast communication network based on a supervised autoencoder, and develops a corresponding hardware concept verification prototype, which is the first prototype of a semantic broadcast communication network. Finally, both GPU simulation and experiments show that the semantic broadcast communication network proposed in the present invention is robust to both channel noise and channel attenuation, and can significantly improve transmission efficiency, fully verifying the network design and its practical feasibility. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments, and the above and / or other advantages of the present invention will become more clear.

[0077] Figure 1 Structure diagram of the semantic broadcast communication framework based on feature decoupling.

[0078] Figure 2 Schematic diagram of the semantic broadcast communication network based on feature decoupling proposed in the present invention.

[0079] Figure 3 Schematic diagram of the semantic sender network and user network.

[0080] Figure 4a is the peak signal-to-noise ratio (PSNR) of the four solutions compared to the test SNR (i.e. SNR test ) Schematic diagram of the test on ANGC.

[0081] Figure 4b is the peak signal-to-noise ratio (PSNR) of the four solutions compared to the test SNR (i.e. SNR test ) on a slow Rayleigh decaying signal.

[0082] Figure 5 Schematic diagram of performance comparison of semantic broadcast communication networks under additive non-Gaussian channels.

[0083] Figure 6 Schematic diagram of performance comparison of semantic broadcast communication networks under Rayleigh fading channels.

[0084] Figure 7Schematic diagram of feature selection performance of the semantic broadcast communication network prototype.

[0085] Figure 8 Schematic diagram of transmission comparison of the semantic broadcast communication network prototype. DETAILED DESCRIPTION

[0086] like Figure 2 As shown, the present invention proposes a method for constructing a semantic broadcast communication network based on feature decoupling, including: establishing a network system framework module, an information theory metric calculation module, and a design module;

[0087] like Figure 1 As shown, the network system framework module includes a semantic source module, a semantic encoder module, a semantic decoder module, a feature selection module, a semantic channel module, and a feature completion module;

[0088] The semantic source module is used to generate original information. The semantic source generates original data X d , usually including expected semantic features and some unexpected semantic features. Since each user may be interested in different functions. Therefore, the semantic sender needs to send the expected semantic features according to the interests of each user, and not send unexpected semantic features.

[0089] The semantic encoder module is used to implement the knowledge base from the data X d Extract L decoupled semantic features from where z l Represents the lth semantic feature

[0090] The feature selection module selects semantic features for each user according to the task requirements. represents the semantic features that user i is interested in, represents the set of serial numbers of the semantic features of interest to user i, then Since user i only has If interested, the remaining features can be regarded as redundant for user i.

[0091] Let X i,s The selected semantic features of user i are given by

[0092]

[0093] In addition, K users X s The selected semantic features of are:

[0094]

[0095] The semantic broadcast channel module is characterized in that the received signal Y of user ii Decode (i=1,2,…,K) and output the estimated semantic features represents the lth estimated semantic feature.

[0096] The feature completion module: Although the unexpected feature of user i If not transmitted, user i may generate corresponding unexpected features by searching the knowledge base. Then by combining the expected features and unexpected features User i can obtain complete semantic features

[0097] The semantic decoder module is characterized by being based on the complete semantic features The semantic decoder module of user i uses the knowledge base to obtain the reconstructed data Finally, the reconstructed data is sent to the semantic destination.

[0098] The information-theoretic metric for semantic broadcast communication is characterized by proposing an optimal distortion allocation scheme for semantic compression of multi-source data and deriving a reachability region for linear semantic communication, where:

[0099] Y i,s =G i X s +N i,s ,i=1,...,K (3)

[0100] Among them, G i represents the effective channel gain of user i from the feature selection module to the feature completion module, N i,s Represents the received semantic noise. The received semantic noise N i,s It follows a non-Gaussian distribution with a variance of

[0101] The distortion distribution in the semantic compression of multi-source data is characterized in that most existing semantic communications focus on the semantic compression of single-source data, while in a multi-user semantic broadcast communication network, each user may be interested in different features, so multi-source data compression is required. Therefore, the following problem arises: how to distribute this distortion to multi-source data so as to achieve the minimum total distortion of the system under distortion and perception constraints.

[0102] The present invention is based on independent source data Among them, L d is the number of independent information sources and L d>1. The goal of the method is to minimize the distortion allocation function R(D,P) of multi-source data compression under the constraints of square error distortion and perception. Specifically, the optimization formula of the distortion allocation function is:

[0103]

[0104] Make

[0105]

[0106] The present invention considers multiple independent Gaussian distribution data in represents a mean of 0 and a variance of Gaussian distribution. Let D i Represents X i,d and The square error distortion between

[0107]

[0108] Based on this, mutual information It can be written as follows:

[0109]

[0110] in, obeys a Gaussian distribution, that is, if [x] + =x, then Otherwise, [x] + =0.

[0111] Therefore, for Gaussian distribution data X i,d and reconstructing data Its KL distance as follows:

[0112]

[0113] Based on the above analysis, the optimal distortion allocation problem (4) can be restated as:

[0114]

[0115] Make

[0116]

[0117] It should be noted that in problem (8) {D i} is convex, and its Lagrangian function L({D i},λ D ,λ P )for:

[0118]

[0119] Among them, λ D ≥0 and λ P ≥0 are the Lagrange multipliers associated with (8b) and (8c), respectively.

[0120] Let the function L({D i},λ D ,λ P ) i The first derivative of is 0:

[0121]

[0122] Therefore, the optimal distortion D i It can be given by the following formula:

[0123]

[0124] Among them, the parameter λ D and λ P is the solution to the following problem:

[0125]

[0126]

[0127] Therefore, data X i,d (i=1,..,L d ) is:

[0128]

[0129] The present invention establishes the reachability region of dual-user degenerate semantic broadcast communication. Where |G1|≥|G2|, σ 1,s =σ 2,s ; G1 and G2 represent the channels of users 1 and 2 respectively; σ 1,s and σ 2,s denote the standard deviation of the noise of user 1 and 2 respectively. Since G1>G2, the semantic feature X of user 2 2,s is encoded into the cloud center signal, and the semantic feature X of user 1 1,s is encoded into the satellite signal. 1,s , R 2,s denote the information rate from the transmitter’s feature selection module to the feature completion module of user 1 and user 2, respectively. If the following conditions are met, the rate pair (R 1,s , R 2,s ) is achievable:

[0130] R 1,s≤I(X 1,s ; Y 1,s |X 2,s ) (14a)

[0131] R 2,s ≤I(X 2,s ; Y 2,s ) (14b)

[0132] Where I(a; b) represents the mutual information between a and b; X 1,s and X 2,s Represent the semantic features selected by user 1 and user 2 respectively; Y 1,s and Y 2,s denote the semantic features estimated by user 1 and user 2 respectively.

[0133] Let P represent the total transmission power of the transmitter, αP and (1-α)P represent the semantic feature X 1,s and (1-α)P allocated power, where α∈[0,1] is the power allocation factor. Therefore, the transmission signal X is:

[0134]

[0135] The signals received by user 1 and user 2 are:

[0136]

[0137]

[0138] User 1 first uses the successive interference cancelation (SIC) method to decode the semantic feature X 2,s , and then cancel X from the received signal 2,s ; Then, user 1 decodes the semantic feature X 1,s After the SIC step, the remaining received signal of user 1 is Given as follows:

[0139]

[0140] User 2 can only decode semantic feature X 2,s , and the semantic feature X 2,s It is interference to user 2.

[0141] The design steps are as follows: First, at the sending end, the encoder module extracts semantic features from the source data and decomposes them into multiple independent and interpretable semantic features. Secondly, through the semantic feature selection module, the sending end broadcasts the semantic features to the desired users. Finally, at the receiving end, when user i receives the expected semantic features, the unexpected features are completed with the help of the knowledge base, and the semantic features are decoded using the semantic decoder module.

[0142] Specifically, the present invention uses a supervised autoencoder (AE) to train a semantic broadcast network in three steps: self-reconstruction, exchange of common features, and exchange of different features. The detailed training steps are as follows:

[0143] Step 1: Self-reconstruction. The semantic encoder module transforms the lth data Compressed into latent semantic feature vector Z (l) :

[0144]

[0145] where l=1,...,L s , L s Represents the total amount of sample data. Indicates data Then, the semantic feature vector Z( l ) is broadcast to K semantic users.

[0146] On the user side, the semantic features received by user i are:

[0147]

[0148] User i decodes the received semantic features And get the reconstructed data as follows:

[0149]

[0150] Finally, the function L is reconstructed based on the standard supervised autoencoder SAE,1 , for the semantic encoder module E φ (·) and the semantic decoder module Perform joint optimization:

[0151]

[0152] Step 2 is to exchange common features. Select two input data with common semantic features and Extract semantic features through the semantic decoder module and (l1,l2=1,...,L s), as follows:

[0153]

[0154]

[0155] set up and The jth semantic features of are the same, and they are exchanged as follows and The jth semantic feature of:

[0156]

[0157] in, represent Data after feature exchange; represent After feature exchange, j=1,...,L c , L c Represents the total number of common semantic features; Express The jth semantic feature and The jth semantic feature of is exchanged; the public semantic feature after exchange and is sent to K users.

[0158] On the receiving end, represents the semantic features received by user 1; Represents the semantic features received by user 2:

[0159]

[0160]

[0161] Through the semantic decoder module User i receives the semantic features The reconstructed data is The received semantic features The reconstructed data is

[0162]

[0163]

[0164] The reconstruction loss function L for common semantic feature exchange AE,2 as follows:

[0165]

[0166] Step 3 is the exchange of different semantic features. Select two input data with different semantic features and l1,l3=1,...,L s , through the semantic encoder module E φ ,extract Semantic features of and Semantic features of As shown below:

[0167]

[0168]

[0169] set up and The nth semantic feature of is different, that is and Different, exchange according to the following formula and The nth semantic feature of:

[0170]

[0171] in, express Semantic features after feature exchange; express Semantic features after feature exchange, n=1,...,L e , L e Represents the total number of different features. and is transmitted to the user end.

[0172] At the receiving end, for user i, let and Respectively represent and Received semantic features:

[0173]

[0174]

[0175] Among them, α i is the equivalent channel gain, N i,s is the equivalent noise.

[0176] Through the semantic decoder module User i receives the semantic features The reconstructed data is The received semantic features The reconstructed data is

[0177]

[0178]

[0179] On this basis, the reconstructed data and As input data, repeat the semantic encoding in (26), exchange different semantic features in (27), semantic feature propagation in (28) and semantic feature decoding in (29).

[0180] After the two semantic features are exchanged, represent The final reconstructed data, represent The final reconstructed data, then the final reconstruction loss L AE,3 for:

[0181]

[0182] The proposed semantic broadcast communication network structure includes a sender and a user, such as Figure 3 As shown in the figure. Conv2D64@64*64 means that the network consists of 64 2-D convolutional filters with an output size of 64×64. Dense 1×4096 means that a fully connected layer has 4096 neurons, and the residual block represents a convolutional block composed of two convolutional networks Conv2D 256@8*8 cascaded. d represents source data; Z represents semantic features; X s represents the semantic features after feature selection; Y i,s represents the semantic features estimated by user i; It represents the semantic features obtained by user i after feature completion; represents the reconstructed data of user i.

[0183] The semantic broadcast network structure proposed in the present invention is as follows, where the symbol “→” represents cascade:

[0184] Semantic sending end network structure:

[0185] User-side network structure:

[0186] Example

[0187] The experimental performance of the proposed semantic broadcast network based on feature decoupling is evaluated through GPU simulation and hardware prototype. The GPU experiment has been carried out on a 502GB RAM Intel Xeon Gold 6240CPU and a 24GB NvidiaGeForce 3090GTX graphics card, and Pytorch is powered by CUDA 11.3. The experiments are carried out on two standard datasets, namely the font dataset and the ilab-20M dataset.

[0188] The robust transmission performance of the proposed semantic broadcast communication scheme is demonstrated over additive non-Gaussian noise channels (ANGC) and slow Rayleigh fading channels. In addition, four broadcast communication transmission schemes are compared, namely, a broadcast communication scheme using JPEG compression in digital communication, a non-robust semantic broadcast communication scheme without adding channel noise during training, and a SNR-based semantic broadcast communication scheme proposed in this invention. train =4dB robust semantic broadcast communication scheme and the proposed SNR train = 8dB robust semantic broadcast communication scheme, where SNR train represents the training SNR of the scheme.

[0189] Figure 4a Describes the peak signal-to-noise ratio (PSNR) of the four solutions compared to the test signal-to-noise ratio (SNR) test The test on ANGC (signal-to-noise ratio SNR, full name SIGNAL-NOISE RATIO), where the additive non-Gaussian noise N 1,s and N 2,s The probability density functions (PDF, ProbabilityDensity Function) are: It can be seen from the figure that the peak signal-to-noise ratio of the JPEG compression scheme is significantly lower than that of the other three schemes. test ≤4dB), when SNR train =4dB, the PSNR of the robust semantic broadcast communication network proposed in the present invention is the highest. train =8dB, the PSNR of the robust semantic broadcast communication network proposed in the present invention is higher than that of the non-robust semantic broadcast system method. For the medium SNR region (4dB≤SNR test ≤8dB), when SNR train = 8dB, the PSNR of the robust semantic broadcast communication method is the highest, which is facing the robustness of the proposed method, especially for the medium and low SNR areas. test≥8dB), the PSNR of the non-robust semantic broadcast communication network is the highest. This is because the impact of noise can be ignored in the high SNR area.

[0190] Figure 4b Describes the peak signal-to-noise ratio (PSNR) of the four schemes compared to the test signal-to-noise ratio (SNR) test Test on Rayleigh decay signal, where Similar to Figure 4a , JPEG compression is the worst among all four methods. When SNR test ≤15dB, the robust semantic broadcast communication network proposed in this invention has an SNR train =4dB and SNR train =8dB, its PSNR is higher than that of the non-robust semantic communication network, which also proves the robustness of the proposed method. test When ≥15dB, the PSNR of non-robust semantic communication is the highest because the impact of noise can be ignored.

[0191] Figure 5 shows the performance comparison for semantic broadcast communication network (SNR for user 1 test =4dB, SNR of user 2 test =8dB). The expected features and knowledge bases of the two users are different. Specifically, as shown in the first two columns, the input data is a red lowercase letter w on a blue background image. The third column shows that the expected features of users 1 and 2 are content and font color, respectively. The fourth column shows that the knowledge bases of users 1 and 2 are green uppercase letters U on a yellow background and yellow uppercase letters E on a gray background, respectively. The fifth column shows the poor transmission performance of the JPEG compression scheme, which cannot recognize the decoded image. The sixth to eighth columns are the non-robust semantic BC scheme, the train =4dB robust semantic BC method and with SNR train = 8dB. The reconstructed data of user 1 is a green lowercase letter w on a yellow background, and the reconstructed data of user 2 is a red uppercase letter E on a gray background, which verifies the effectiveness of the feature selection and feature completion of the proposed semantic BC network.

[0192] also, Figure 5The input data of the last two rows is a white car with a lower left pose image, and the third column shows that the expected features of users 1 and 2 are content and pose, respectively. The fourth column shows that the knowledge bases of users 1 and 2 are red, white, and green left pose cars and red pickup trucks with upper right pose, respectively. Similarly, the fifth column shows that the decoded images of the JPEG compression scheme are unrecognizable. The sixth to eighth columns show that the reconstructed data of users 1 and 2 include the expected features and the features of the user knowledge base, where only the expected features are transmitted. This also verifies the effectiveness of feature selection and feature completion of the proposed semantic broadcast communication network. Figure 6 shows the performance comparison of the four schemes on the Rayleigh fading channel, similar to Figure 5 ,The results demonstrate the effectiveness of feature selection and ,feature completion of the proposed semantic broadcast communication network.

[0193] Figure 7 shows the feature selection performance in a semantic broadcast communication network, where the expected features and knowledge bases of users 1 and 2 are different. The input data of the first two rows is a small blue italic letter F, and the third column shows that the expected features of users 1 and 2 are content and style, respectively. The fourth column shows that the knowledge bases of users 1 and 2 are a large pink letter U in a regular font and a large red letter T in a regular font, respectively. The fifth column shows that the reconstructed image of user 1 is a large pink letter F in a regular font, and the reconstructed data of user 2 is a small italic red letter T. Therefore, only the expected semantic features are transmitted to the users, while unexpected semantic features are generated based on the knowledge base. Similarly, columns 3 to 5 also verify the effectiveness of feature selection and feature extraction.

[0194] Figure 8 The original image transmission scheme, JPEG compression scheme, non-robust semantic broadcast communication scheme and SNR are given. train =8dB transmission time, compression ratio, peak signal-to-noise ratio and reconstruction data of the robust semantic broadcast communication method. Figure 8 It shows that the transmission time of the non-robust semantic broadcast communication scheme and the robust semantic broadcast communication method is significantly lower than that of the JPEG compression scheme (168.69ms) and the original image transmission scheme (466.44ms). In addition, the compression ratio of the non-robust semantic broadcast communication scheme and the robust semantic broadcast communication algorithm is 1966.09, which is significantly higher than that of the JPEG compression scheme (21.22) and the original image transmission scheme. Therefore, the proposed network can significantly reduce the transmission load and transmission time.

[0195] In a specific implementation, the present application provides a computer storage medium and a corresponding data processing unit, wherein the computer storage medium can store a computer program, and when the computer program is executed by the data processing unit, the invention content of a method for constructing a semantic broadcast communication network based on feature decoupling provided by the present invention and some or all of the steps in each embodiment can be run. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0196] Those skilled in the art can clearly understand that the technical solutions in the embodiments of the present invention can be implemented by means of computer programs and their corresponding general hardware platforms. Based on this understanding, the technical solutions in the embodiments of the present invention can be essentially or partly contributed to the prior art in the form of a computer program, i.e., a software product, which can be stored in a storage medium and includes several instructions for enabling a device including a data processing unit (which can be a personal computer, a server, a single-chip microcomputer, a MUU or a network device, etc.) to execute the methods described in various embodiments of the present invention or certain parts of the embodiments.

[0197] The present invention provides a method for constructing a semantic broadcast communication network based on feature decoupling. There are many methods and ways to implement the technical solution. The above is only a preferred implementation of the present invention. It should be pointed out that for ordinary technicians in this technical field, without departing from the principle of the present invention, several improvements and modifications can be made, and these improvements and modifications should also be regarded as the protection scope of the present invention. All components not specified in this embodiment can be implemented by existing technologies.

Claims

1. A method for constructing a semantic broadcast communication network based on feature decoupling, characterized in that: include: Establish network system framework module, information theory metric calculation module, and design module; The network system framework module is used to extract semantic features and transmit semantic features; The information theory metric calculation module is used to construct an optimal distortion allocation method for multi-source data semantic compression and establish a reachability region; The design module is used to construct a lightweight and robust semantic broadcast communication network based on supervised autoencoders to achieve controllable decoupling of semantic features; Each user has a semantic decoder module based on the complete semantic features. The semantic decoder module of user i uses the knowledge base to obtain the reconstructed data Finally, the data will be reconstructed Send to semantic destination; The design module is used to construct a lightweight and robust semantic broadcast communication network based on a supervised autoencoder, which specifically includes the following steps: Step 1, self-reconstruction training: semantic encoder module E φ (·) Use the following formula to convert the lth data Compressed into a latent semantic feature vector Z (l) : where l=1,...,L s , L s Represents the total amount of data; Indicates data Perform feature extraction; Semantic feature vector Z (l) is broadcast to K semantic users; at the user end, the received semantic features for: where i = 1, ..., K; G i Represents the effective channel gain; N i,s represents the semantic noise received by user i; User i decodes the received semantic features And get the reconstructed data as follows: in, Indicates the received semantic features Decode to obtain reconstructed data Reconstruction loss function L based on standard supervised autoencoder AE,1 , for the semantic encoder module E φ (·) and the semantic decoder module Perform joint optimization: Step 2: Further train the lightweight robust semantic broadcast communication network by exchanging common semantic features and minimize the reconstruction loss of common semantic feature exchange: Select two input data with common semantic features and Extracted by semantic decoder module Semantic features of and Semantic features of l1,l2=1,...,L s ;set up The jth semantic feature of and The jth semantic feature of Same, exchange according to the following formula and The jth semantic feature of: in, represent Data after feature exchange; represent After feature exchange, j=1,...,L c , L c Represents the total number of common semantic features; Indicates that The jth semantic feature and The jth semantic feature of and is sent to K users; At the receiving end, represents the semantic features received by user 1; Represents the semantic features received by user 2; using the semantic decoder module Reconstructed data and Reconstructed data as follows: Reconstruction loss function L for common semantic feature exchange AE,2 as follows: Step 3: further train the lightweight robust semantic broadcast communication network by exchanging different semantic features and minimize the reconstruction loss of exchanging different semantic features; The network system framework module includes a semantic source module, a semantic encoder module, a semantic decoder module, a feature selection module, a semantic broadcast channel module, and a feature completion module; The semantic source module is used to generate original data, and the generated original data X d Including expected semantic features and unexpected semantic features; the sender needs to send expected semantic features according to the interests of each user, but does not need to send unexpected semantic features; The semantic encoder module is used to: Based on the knowledge base, from the original data X d Extract L decoupled semantic features from where z l represents the lth semantic feature, Defined as a set of semantic feature numbers, that is The feature selection module is used to select semantic features for each user according to task requirements: represents the semantic features that user i is interested in, represents the set of serial numbers of the semantic features that user i is interested in, then Since user i only has If interested, the remaining features is regarded as the redundancy of user i; where z n represents redundant features, Redundant features are the semantic features that users are interested in removed from the total semantic features; Let X i,s represents the selected semantic features of user i as follows: Then the selected semantic features X of K users s for: The semantic broadcast channel module receives the signal Y from user i. i Decode, i = 1, 2, ..., K, and output the semantic features estimated by user i represents the lth estimated semantic feature; with input X s and output (Y 1,s ,…,Y K,s ) is represented as the conditional probability p(y 1,s ,…,y K,s |X s ), where y K,s Represents the signal variable received by the Kth user; The feature completion module is used to: Not transmitted, user i automatically generates corresponding unexpected features by searching the knowledge base Finally, by combining the expected features and unexpected features User i can obtain complete semantic features The information theory metric calculation module converts the conditional probability p(y 1,s ,…,y K,s |X s ) is set to satisfy the following relationship: Y i,s =G i X s +N i,s (2) Among them, G i represents the effective channel gain of user i from the feature selection module to the feature completion module, N i,s represents the semantic noise received by user i; set the received semantic noise N i,s It follows a non-Gaussian distribution with a variance of 2. According to claim 1, a method for constructing a semantic broadcast communication network based on feature decoupling is characterized in that: The optimal distortion allocation method for multi-source data semantic compression is based on independent source data The optimal distortion allocation method is d is the number of independent information sources and L d >1, the goal of the method is to minimize the distortion allocation function R(D, P) of multi-source data compression under the constraints of square error distortion and perception. Specifically, the optimization formula of the distortion allocation function R(D, P) is: So that: Among them, the function min(·) means finding the minimum value; I(a; b) means the mutual information between a and b; E(·) means finding the expectation; Represents the source data X i,d and estimated values ​​of source data The distance between them; D represents the threshold of total distortion; d i (·) represents the relative entropy KL distance; Representation data Distribution of Represents data X i,d Distribution of Represents an estimate of the source data X-based i,d The conditional probability distribution of ; represents the set of estimated values ​​of source data; P represents the threshold of relative entropy; Consider multiple independent Gaussian distribution data in represents a mean of 0 and a variance of Gaussian distribution; let D i Represents X i,d and The squared error distortion between is as follows: Among them, the parameter λ D and λ P is the solution to the following problem:

3. According to claim 2, a method for constructing a semantic broadcast communication network based on feature decoupling is characterized in that: DataX i,d The rate-distortion function R i (i=1,..,L d )for:

4. The method for constructing a semantic broadcast communication network based on feature decoupling according to claim 3 is characterized in that: The information theory metric calculation module is also used to establish the reachability region of dual-user degenerate semantic broadcast communication: for dual users, |G1|≥|G2|, σ 1,s =σ 2,s ; G1 and G2 represent the channel of user 1 and user 2 respectively; σ 1,s and σ 2,s They represent the noise standard deviation of user 1 and user 2 respectively.

5. A method for constructing a semantic broadcast communication network based on feature decoupling according to claim 4, characterized in that: The design module also performs the following steps: φ (·) represents the semantic encoder module, i.e., the feature extraction network, and the subscript φ represents the neural network parameter set of the semantic encoder module; first, at the sending end, E φ (·) Extract semantic features from the source data and decompose them into two or more independent and interpretable semantic features; secondly, through the semantic feature selection module, the sender broadcasts the semantic features to the expected user; finally, at the receiving end, when user i receives the expected semantic features, the knowledge base is used to complete the unexpected features and the semantic decoder module is used to decode the expected features. Decode the semantic features, where represents the semantic decoder of the i-th user, i=1,...,K, subscript θ i Represents the set of neural network parameters of the semantic decoder module of user i.

6. A method for constructing a semantic broadcast communication network based on feature decoupling according to claim 5, characterized in that: Step 3 includes: selecting two input data with different semantic features and where l1,l3=1,...,L s , through the semantic encoder module E φ ,extract Semantic features of and Semantic features of set up The nth semantic feature of and The nth semantic feature of Different, exchange according to the following formula and The nth semantic feature of: in, express Semantic features after feature exchange; express Semantic features after feature exchange, n=1,...,L e , L e Represents the total number of different features; after exchange and Be transmitted to the user end; On the receiving end, Receiving The semantic features of Receiving Semantic features: Among them, α i is the equivalent channel gain, N i,s is the equivalent noise; Through the semantic decoder module User i receives the semantic features The reconstructed data is The received semantic features The reconstructed data is Reconstruct the data and As input data, repeat the semantic encoding in (14), the exchange of different semantic features in (15), the propagation of semantic features in (16), and the decoding of semantic features in (17); After the two semantic features are exchanged, Represents input data The final reconstructed data, Represents input data The final reconstructed data, then the final reconstruction loss L AE,3 for:

Images