Semantic communication system resource allocation method for image recovery task

Abstract

The invention discloses an image recovery task-oriented semantic communication system resource allocation method, which comprises the following steps of: considering that a base station and a plurality of users exist in a cellular cell, and sending a pilot signal to the base station by each user; the base station carries out channel estimation; carrying out uplink channel signal to interference plus noise ratio and signal to interference plus noise ratio statistical characteristic analysis; analyzing transmission energy consumption of the semantic information sent by the user through a signal to interference plus noise ratio; calculating the theoretical average error symbol performance of an uplink channel of each user based on the statistical characteristics of the signal to interference plus noise ratio; obtaining CPU periodicity mapping curves required by the user side to extract the semantic information and the base station side to recover the image information under different semantic compression ratios, and calculating the calculation energy consumption of the user side to extract the semantic information and the base station side to recover the image information through a curve fitting method; an optimization problem is obtained, and a final semantic resource allocation problem needing to be optimized is obtained through simplification; a new deep reinforcement learning algorithm is designed and established based on a multi-agent near-end strategy optimization algorithm to solve the resource allocation problem. The advantages of the proposed algorithm are verified through comparison with baseline algorithms such as a multi-agent near-end strategy optimization algorithm and the like.

Description

Technical Field

[0001] The invention belongs to the field of communication technology, and in particular to a semantic communication system resource allocation method oriented to image restoration tasks. Background Art

[0002] With the acceleration of 5G commercialization, 6G has gradually become the direction of global countries' deployment. In recent years, under the strong promotion and support of the Chinese government and driven by the strategic needs of the country's medium- and long-term development, in order to maintain and expand China's leading edge in 5G, China is actively exploring the technical route of independent innovation in 6G. 6G networks are expected to connect people and machines with different levels of intelligence, and semantic communication is one of the key technologies of the current 6G candidates. The semantic communication system mainly focuses on the semantic representation, transmission and reconstruction of the source content, as well as semantic-based wireless transmission. Further consideration, when there are multiple users at the sending or receiving end of the semantic communication system at the same time, a semantic communication network is formed, and the scarcity of wireless resources in the network will make communication resource management very important. Therefore, it is urgent to study resource allocation strategies that can both improve the overall performance of the network and support high-quality semantic services. Through reasonable communication resource allocation, users can be provided with better communication services, such as higher data rates, larger system capacity, and better user experience. Therefore, the reasonable allocation of communication resources is the most direct and effective means to solve the shortage of communication resources in a complex communication environment. Summary of the invention

[0003] The purpose of the present invention is to provide a new MAPPO-based semantic communication system resource allocation method, which solves the problem of power allocation, semantic compression rate and computing capacity allocation of cell semantic communication system by utilizing the proposed LSTM-NOISE MAPPO semantic communication resource algorithm, taking into account the dynamic changes of user location and semantic communication physical channel.

[0004] The technical solution to achieve the purpose of the present invention is: on the one hand, a semantic communication system resource allocation method for image restoration tasks is provided, the method comprising:

[0005] Step 1: Establish a semantic communication system, including a base station and a number of users randomly distributed within the coverage of the base station signal; each user stores a neural network for extracting image semantic information, and an edge computing node is installed at the base station, which stores a neural network for restoring images using semantic information;

[0006] Step 2, the user sends a pilot signal to the base station, and different pilot signals are orthogonal to each other;

[0007] Step 3, the base station performs channel estimation on the uplink channel of each user based on the pilot signals sent by all users;

[0008] Step 4: Based on the uplink channel estimation result, a theoretical analysis is performed on the signal to interference plus noise ratio (SINR) of the transmitted signal at the receiving end and the statistical characteristics of the SINR;

[0009] Step 5, using the SINR of each user to analyze the transmission energy consumption of the user sending semantic information, and calculating the theoretical average symbol error performance of the signal sent by each user according to the modulation mode;

[0010] Step 6, obtaining a mapping curve of the number of CPU cycles required for the user to extract semantic information and the base station to restore image information at different semantic extraction rates, performing numerical fitting on the mapping curve, and calculating the computational energy consumption for extracting semantic information and restoring image information based on the fitting result;

[0011] Step 7, construct an optimization problem, and simplify the problem to obtain the semantic communication resource allocation problem that ultimately needs to be optimized;

[0012] Step 8: Design and build a new deep reinforcement learning algorithm (LSTM-NOISE MAPPO semantic communication resource allocation algorithm) based on the multi-agent proximal policy optimization algorithm (MAPPO) to solve the semantic communication resource allocation problem. During the training process, a dynamically changing training environment is constructed by updating the user's position and channel state in each interaction round to train the user offline until the training results converge.

[0013] On the other hand, a semantic communication system resource allocation system for image restoration tasks is provided, the system comprising:

[0014] The first module is used to establish a semantic communication system, including a base station and a number of users randomly distributed within the coverage of the base station signal; each user stores a neural network for extracting image semantic information, and the base station is equipped with an edge computing node, which stores a neural network for restoring images using semantic information;

[0015] A second module is used for the user to send a pilot signal to the base station, and different pilot signals are orthogonal to each other;

[0016] A third module is used for the base station to perform channel estimation on the uplink channel of each user based on the pilot signals sent by all users;

[0017] The fourth module is used to perform theoretical analysis on the SINR and the statistical characteristics of the SINR of the transmitted signal at the receiving end based on the uplink channel estimation result;

[0018] The fifth module is used to analyze the transmission energy consumption of semantic information sent by each user by using the SINR of each user, and calculate the theoretical average symbol error performance of the signal sent by each user according to the modulation mode;

[0019] The sixth module is used to obtain a mapping curve of the number of CPU cycles required for the user to extract semantic information and the base station to restore image information under different semantic extraction rates, perform numerical fitting on the mapping curve, and calculate the computational energy consumption for extracting semantic information and restoring image information based on the fitting results;

[0020] The seventh module is used to construct an optimization problem and simplify the problem to obtain the semantic communication resource allocation problem that ultimately needs to be optimized;

[0021] The eighth module is used to design and build a new deep reinforcement learning algorithm based on a multi-agent proximal strategy optimization algorithm to solve the semantic communication resource allocation problem. During the training process, a dynamically changing training environment is constructed by updating the user's position and channel status in each interaction round to train the user offline until the training results converge.

[0022] Compared with the prior art, the present invention has the following significant advantages:

[0023] (1) The present invention performs channel estimation in the construction of the communication system and analyzes and obtains a closed-form expression for the average symbol error performance of the user communication link, constructs a semantic communication theoretical model that is closer to the actual communication process, and further lays the foundation for users to adaptively adjust resource allocation strategies during offline training.

[0024] (2) The proposed LSTM-NOISE MAPPO semantic communication resource allocation algorithm can solve both discrete resource allocation strategies and continuous resource allocation strategies. At the same time, the algorithm of the present invention can not only solve the single-modal semantic communication resource allocation problem, but also be applied to the solution of multi-modal and multi-task semantic communication resource allocation problems.

[0025] (3) The proposed LSTM-NOISE MAPPO semantic communication resource allocation algorithm solves the problem of poor training effect of the MAPPO algorithm caused by dynamic changes in the environment. The advantages of the proposed algorithm over the MAPPO algorithm are specifically reflected in:

[0026] This algorithm can bring better data utilization to the system than the MAPPO algorithm, which is manifested in that it has fewer training convergence steps;

[0027] This algorithm can bring better energy consumption stability to the system than the MAPPO algorithm. Specifically, after the training converges, it can make the system have a smaller energy consumption variance while achieving the same training effect as the MAPPO algorithm.

[0028] This algorithm can bring better delay stability to the system than the MAPPO algorithm. Specifically, after the training converges, it can make the system have a smaller delay variance while achieving the same training effect as the MAPPO algorithm.

[0029] This algorithm can bring better symbol error performance to the system than the MAPPO algorithm. Specifically, after training convergence, the system can obtain a smaller average maximum symbol error rate and variance.

[0030] The present invention is further described in detail below in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is a specific flow chart of the semantic communication system resource allocation method for image restoration tasks of the present invention.

[0032] Figure 2 It is a schematic diagram of a cell semantic communication system involved in the present invention.

[0033] Figure 3 It is a schematic diagram of the image sending and recovery process.

[0034] Figure 4 This is a comparison chart of the system energy consumption of the algorithm of the present invention and other different algorithms. Figure 4 (a) to (d) are the system energy consumption of the algorithm of the present invention, the LSTM MAPPO algorithm, the NOISE MAPPO algorithm and the MAPPO algorithm respectively.

[0035] Figure 5 is a comparison chart of system delays between the algorithm of the present invention and other different algorithms, where Figure 5 (a) to (d) are the system delays of the algorithm of the present invention, the LSTM MAPPO algorithm, the NOISE MAPPO algorithm and the MAPPO algorithm respectively.

[0036] Figure 6 This is a comparison chart of the system link rate performance of the algorithm of the present invention and other different algorithms. Figure 6 (a) to (d) are the system link rate performances of the algorithm of the present invention, the LSTM MAPPO algorithm, the NOISE MAPPO algorithm and the MAPPO algorithm respectively. DETAILED DESCRIPTION

[0037] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0038] It should be noted that if there are descriptions involving "first", "second", etc. in the embodiments of the present invention, the descriptions of "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0039] In one embodiment, in combination Figure 1 , provides a semantic communication system resource allocation method for image restoration tasks, the method comprising:

[0040] Step 1: Establish a semantic communication system, including a base station and several users randomly distributed within the coverage of the base station signal; each user stores a neural network for extracting image semantic information, and the base station is equipped with an edge computing point (ECP), which stores a neural network for restoring images using semantic information; Figure 2 As shown, in the center of the cell, there is a r The base station (BS) with 10 antennas and an edge computing point (ECP) provides semantic communication services for M single-antenna users randomly scattered in the cell (N r ≥M), the device set is defined as

[0041] Step 2, the user sends a pilot signal to the base station, and different pilot signals are orthogonal to each other;

[0042] Step 3, the base station performs channel estimation on the uplink channel of each user based on the pilot signals sent by all users;

[0043] Step 4: Based on the uplink channel estimation result, a theoretical analysis is performed on the signal to interference plus noise ratio (SINR) of the transmitted signal at the receiving end and the statistical characteristics of the SINR;

[0044] Step 5, using the SINR of each user to analyze the transmission energy consumption of the user sending semantic information, and calculating the theoretical average symbol error performance of the signal sent by each user according to the modulation mode;

[0045] Step 6, obtaining a mapping curve of the number of CPU cycles required for the user to extract semantic information and the base station to restore image information at different semantic extraction rates, performing numerical fitting on the mapping curve, and calculating the computational energy consumption for extracting semantic information and restoring image information based on the fitting result;

[0046] Step 7, construct an optimization problem, and simplify the problem to obtain the semantic communication resource allocation problem that ultimately needs to be optimized;

[0047] Step 8: Design and build a new deep reinforcement learning algorithm based on the multi-agent proximal policy optimization algorithm (Multi-Agent Proximal Policy Optimization, MAPPO) to solve the semantic communication resource allocation problem. During the training process, a dynamically changing training environment is constructed by updating the user's position and channel state in each interaction round to train the user offline until the training results converge.

[0048] Furthermore, in one embodiment, step 1 further includes:

[0049] Before executing the semantic communication task, the semantic transceiver network of the semantic communication sender and receiver is trained for the image restoration task. After the training, the transceiver network parameters corresponding to the specific semantic background knowledge are obtained; the semantic transceiver network includes a neural network for extracting image semantic information at the user end and a neural network for restoring the image using the semantic information at the edge computing node. After the semantic transceiver network training is completed, the user and the base station perform the image restoration task under the specific semantic background knowledge through semantic communication. The schematic diagram of the above training process and the process of executing the communication task is shown in the figure. Figure 3 shown.

[0050] Consider using an automatic codec as the structure of the transceiver network. The semantic background knowledge base used in training is associated with the specific semantic task being performed, which may include image background knowledge in specific application fields such as medicine, transportation, and chemical engineering. After the training is completed, the transceiver network parameters corresponding to the specific semantic background knowledge are obtained.

[0051] Further, in one of the embodiments, in step 2, at the beginning of each channel coherence interval, M single-antenna users in the cell simultaneously send mutually orthogonal pilot signals to the base station, and the number of antennas of the base station is represented by N r , the number of pilot signals is M, and the pilot dimension size is 1×τ p , the pilot vector sent by the kth user is represented by Φ k , whose elements are represented by satisfy τ p Greater than or equal to M, the transmission power of the pilot signal is the maximum transmission power of the user, expressed as p kIn order to save pilot overhead, we can let τ p =M.

[0052] Further, in one embodiment, step 3 specifically includes:

[0053] make N represents the number of base stations in the cell r The uplink fading between the root antenna and the kth user is expressed as:

[0054]

[0055] In the formula, h k YesN r ×1 vector, each element of which follows a complex Gaussian distribution with a mean of 0 and a variance of 1, indicating the small-scale fading between the user and each antenna of the base station; β k is a scalar, representing the large-scale fading between the kth user and the base station;

[0056] After all users send pilot signals, the received signal at the base station is represented by Y:

[0057]

[0058] In the formula, the dimension of Y is N r ×M, M is the number of pilot signals, n is the dimension N r ×M complex additive Gaussian noise matrix, whose elements satisfy independent and identical distribution CN(0,σ 2 ), CN(0,σ 2 ) means the mean is 0 and the covariance is σ 2 The complex Gaussian distribution of is obtained by the orthogonality between different pilot vectors. Φ j represents the pilot signal sent by the jth user, and H represents the conjugate transpose;

[0059] make Multiply both sides of the equal sign of formula (2) on the right to obtain:

[0060]

[0061] In the formula, The dimension size is N r ×1;

[0062] Based on formula (3), the MMSE method is used to estimate the channel and the uplink channel of the kth user is obtained as

[0063]

[0064] Further, in one embodiment, step 4 specifically includes:

[0065] (1) After channel estimation, the received signal of the base station during the communication process is expressed as y:

[0066]

[0067] In the formula, x k represents the transmitted signal of the kth user, p′ k represents the signal transmission power of the kth user,

[0068] For any k-th user, the base station side calculates the dimension of 1×N r The weight vector v k The inner product of the received signal y is used to linearly combine the received signal, which can be expressed as:

[0069]

[0070] Where p′ k' represents the signal transmission power of the k'th user, x k' represents the transmitted signal of the k'th user, represents the channel estimation value of the k'th user,

[0071] Based on formula (6), the maximum SINR of the kth user is expressed as γ k :

[0072]

[0073] In the formula, The dimension is N r ×N r The identity matrix, C k′ is the mean square value of the channel estimation error of the k′th user, expressed as:

[0074]

[0075] In the formula, β k' is a scalar, representing the large-scale fading between the k′th user and the base station;

[0076] For the kth user, the general closed-form expression of the uplink channel SINR distribution has high complexity. When the number of antennas configured in the system is large, it is difficult to obtain an accurate closed-form solution. Therefore, an asymptotic method is used to analyze the probability density function of the uplink channel SINR of any user in this system.

[0077] (3) Analyze the probability density function of the uplink channel SINR of any user by asymptotic method

[0078] Let matrix The eigenvalues ​​of converge to λ1. According to the Marchenko-Pastur distribution criterion, the mean of λ1 is calculated as:

[0079]

[0080] Therefore, formula (7) is simplified to:

[0081]

[0082] In the formula, represents the jth element of the channel estimation vector of the kth user, and the expression of φ is as follows:

[0083]

[0084] The right side of formula (10) consists of two terms, each of which can be regarded as the sum of exponential random variables with the same mean, which obeys the chi-square distribution. Therefore, the statistical distribution of SINR in formula (10) is characterized by the convolution of the two distributions on the right side of the equation, expressed as:

[0085]

[0086] In the formula,

[0087]

[0088] N1=M-1

[0089]

[0090] N2=N r -M+1

[0091]

[0092] Further, in one embodiment, step 5 specifically includes:

[0093] For various digital modulation systems, such as PAM, PSK, and QAM, the average symbol error rate is expressed by the Gaussian Q function:

[0094]

[0095] Among them, the Gaussian Q function is expressed as:

[0096]

[0097] Parameters a and b are constants determined by the specific modulation scheme;

[0098] Substituting formula (14) into formula (13), we get:

[0099]

[0100] Among them, the closed expression of C(x) is as follows:

[0101]

[0102] in,

[0103] Substituting C(x) into formula (15) we get expression.

[0104] Further, in one embodiment, step 6 specifically includes:

[0105] For each user, the energy consumption consists of two parts: the energy consumption of extracting semantic information E 1k And the energy consumption of uplink transmission E 2k ;

[0106] (1) The energy consumption E of extracting semantic information 1k for:

[0107] E 1k =κy 1k (D k ,S k )f 2 (17)

[0108] In the formula, y 1k (D k ,S k ) represents the number of CPU cycles for extracting semantic information of the kth user in the cell, D k and S k They represent the data size before and after semantic extraction for the kth user, respectively. The size value is related to the semantic extraction rate ρ k f represents the computing capacity of the kth user, and the local computing capacity of each user in the system is assumed to be equal; κ represents the energy consumption coefficient;

[0109] The numerical fitting method is used to obtain y 1k (D k ,S k ) and ρ k The relationship between them is expressed as follows:

[0110] y 1k (D k ,S k )=η1ρ k +η2 (18)

[0111] In the formula, η1>0 and η2>0 are constant parameters respectively; it should be noted that the numerical fitting result obtained by formula (18) is obtained under the semantic coding model selected by the present invention and the simulation software and hardware conditions, and the numerical fitting research under different semantic coding models and software and hardware conditions should be specifically analyzed.

[0112] The time it takes for a user to extract semantic information is:

[0113]

[0114] (2) Energy consumption E of the uplink transmission 2k for:

[0115] E 2k =t 2k p k ′(20)

[0116] Among them, the time required for uplink transmission is t 2k :

[0117]

[0118] In the formula, Cap k Cap represents the uplink channel capacity of the kth user, which is expressed as k =Wlog2(1+γ k ), where W is the uplink bandwidth allocated to each user, S k The amount of data after extracting semantic information for the kth user;

[0119] (3) For edge computing nodes, the energy consumption is the computational energy consumption when performing image restoration on the semantic information sent by users in the cell, expressed as:

[0120]

[0121] In the formula, E 3k represents the energy consumption of decoding the information sent by the kth user, It represents the number of CPU cycles required to decode the information sent by the kth user. and They represent the amount of semantic information data after transmission through the wireless channel and the amount of data after semantic decoding. Since the neural network structure of the semantic encoding and semantic decoding process is symmetrical, it can be considered that f 3k represents the computing capacity allocated to the kth user;

[0122] The corresponding time consumption is:

[0123]

[0124] In summary, the total energy consumption E of the kth user to perform his semantic communication task is k Expressed as: E k =E 1k +E 2k +E 3k .

[0125] Furthermore, in one of the embodiments, the system goal is to minimize the total energy consumption of all users in the semantic communication system by adjusting the power, semantic compression rate and computing capacity allocation of each user under the semantic communication task, so the optimization problem constructed in step 7 is:

[0126]

[0127] The optimization problem is a mixed integer nonlinear programming problem, where the constraint C1 is the delay constraint for each user in the cell to perform the semantic communication task, C2 represents the accuracy constraint for each user to perform the semantic task, and t th represents the upper limit of the delay constraint for each user, u k (D k ,S k ) represents the semantic accuracy of semantic communication of the kth user, A k represents the lower limit of the semantic accuracy constraint of the kth user, C3 and C4 are the computing capacity constraints of the edge computing nodes, and F max represents the maximum computing capacity of the edge computing node, C5 is the signal transmission power constraint for each user, and p level1 , p level2 , p level3 , p level4 They represent the optional power, C6 is the link symbol error performance constraint for each user, is the upper limit of the symbol error performance constraint for each user, C7 represents the sum constraint of the link rate of all users in the cell, Γ th Indicates the lower limit of the link rate sum constraint,

[0128] In the process of semantic communication, since the higher the semantic extraction rate, the more information the transmitted signal carries, it can be considered that there is a positive correlation between the execution accuracy of the semantic task and the semantic extraction rate. Therefore, the optimization problem P1 can be equivalent to:

[0129]

[0130]

[0131] Where C2 constraint is the constraint condition of each user’s semantic extraction rate, ρth Represents the lower limit of the semantic extraction rate constraint for each user.

[0132] Further, in one embodiment, step 8 specifically includes:

[0133] For the optimization problem P2, the overall solution idea is: decompose the problem into two sub-problems, and solve the two sub-problems in turn to make P2 converge to the optimal value;

[0134] First, we fix the variable ρ k and p′ k When P2 is transformed into a single variable f 3k The sub-problem is expressed as follows:

[0135]

[0136] P3 is a convex optimization problem, which is solved using the convex optimization tool CVXPY;

[0137] Considering each user as an intelligent agent in deep reinforcement learning, under the condition that information can be shared among all users, the observed state of the kth user in the tth training round is defined as o k (t)@{Task k (t),Energy(t),Max_Delay(t),Max_SER(t),Action - (t)}, where Task k (t) represents the size of the original data that the kth user needs to transmit in the tth training round, Energy(t) represents the total energy consumption of all users in the t-1th training round, Max_Delay(t) represents the maximum value of the delay experienced by all users in the t-1th training round, Max_SER(t) represents the maximum value of the average symbol error rate of all user links in the t-1th training round, and the action of the kth user is defined as a k (t)@{p k (t),ρ k (t)}, where p k (t) represents the power selection scheme of the user in the tth training round, ρ k (t) represents the semantic extraction rate selection scheme of the user in the tth training round, Action - (t) represents the set of actions taken by users other than the user himself in the t-1th training round, that is, Action - (t)={a1(t)∪a2(t)...a k-1 (t)∪a k+1 (t)...∪a M (t)};

[0138] The training adopts a centralized training method, defining the global state as the set of all user states, that is,

[0139] The reward value of each user in the training round is expressed as: The second, third, and fourth items on the right side of the equation r(t) are penalty items. λ1, λ2, λ3, and λ4 are weight coefficients of the corresponding items. All users share the same reward value. In each round, each user uses their own strategy network to output actions according to their own observation state and uploads their own state to the base station. The edge computing node at the base station calculates r(t) and uses the evaluation network to output the global state value function V after obtaining the global state information. ζ (s(t)), where ζ represents the parameters of the evaluation network.

[0140] Furthermore, in one embodiment, the Long short-term memory-NoiseMAPPO (LSTM-NOISEMAPPO) algorithm proposed in step 8 is designed based on the MAPPO algorithm. The algorithm is based on the user strategy network structure and the evaluation network output V ζ There are innovations in the process of (s(t)), as follows:

[0141] Benefiting from the theoretical design of the policy network and evaluation network parameter update mechanism, the MAPPO algorithm has strong training stability and optimization expectations. However, in the face of a changing training environment, the MAPPO algorithm often causes large fluctuations in training results due to unknown random factors in the environment. There are two possible reasons for this phenomenon:

[0142] On the one hand, the policy network structure of MAPPO generally uses a multilayer perceptron (MLP) neural network, which only focuses on the user's observation state in each current training round and does not care about the observation state experienced by the user in the past. Therefore, when faced with a new observation state, the policy network cannot extract effective information from the observation state experienced in the past to output actions for the intelligent agent that can help stabilize or improve the training effect.

[0143] On the other hand, the evaluation network has poor generalization ability in dynamic environments and cannot give appropriate output V when a joint state that has not been evaluated before appears in the environment. ζ (s(t)), resulting in incorrect policy network parameter updates.

[0144] Considering the above two reasons, this algorithm improves the MAPPO algorithm as follows under the premise of limited energy storage of user nodes:

[0145] The new deep reinforcement learning algorithm based on the multi-agent proximal policy optimization algorithm in step 8 is to improve the multi-agent proximal policy optimization algorithm MAPPO, specifically including:

[0146] (1) The set of state values ​​of all training rounds in each user's game (episode) is regarded as a time series. The policy network of each user is designed as a long short-term memory neural network (LSTM) that can extract time series features to replace the MLP network structure in the policy network of the MAPPO algorithm. The user's policy in each training round is obtained by calculating the user's observed state through the LSTM network.

[0147] (2) In the evaluation function, i.e., the global state value function V ζ In the output process of (s(t)), noise is added to the global state s(t), and the global state with noise added is It is expressed as follows:

[0148]

[0149] In the formula, concat(·) function represents the concatenation function, which is used to horizontally concatenate s(t) and the noise matrix x(t). is the global state after adding noise, each element in x The mean is 0 and the variance is Normal distribution, ε is the scaling factor, ε:U(a,b), U(a,b) represents a uniform distribution between a and b; in addition, a noise factor is added in the calculation process of the advantage function;

[0150] In the tth round of the user offline training phase, the policy network of the kth user is based on the observed state o of the user. k (t) Choose the appropriate action a k (t), the action output process is expressed as follows:

[0151] Forget gate: determines which information in the cell state at the previous moment needs to be forgotten; the activation value f of the forget gate t From the current input o k (t) and the hidden state h′ at the previous moment t-1 It is calculated through a fully connected layer with a sigmoid activation function, W if , W hf 、b if 、b hf Parameters of the fully connected layer:

[0152] f t =σ(W if ok (t)+b if +W hf h′ t-1 +b hf ) (28)

[0153] Input gate: determines which information in the current input should be added to the cell state. It includes two parts: first, the sigmoid function is used to determine the information acceptance weight i t , and secondly, the candidate state is calculated by the tanh function, g t , W ii , W hi 、b ii 、b hi , W ig , W hg 、b ig 、b hg They are the corresponding fully connected layer parameters:

[0154] i t =σ(W ii o k (t)+b ii +W hi h′ t-1 +b hi ) (29)

[0155] g t =tanh(W ig o k (t)+b ig +W hg h′ t-1 +b hg ) (30)

[0156] Cell state update: Combine the results of the forget gate and the input gate to update the cell state c′ t :

[0157] c′ t =f t *c t-1 +i t *g t (31)

[0158] In the formula, * represents the bitwise multiplication operation, c t-1 Indicates the cell state at the previous moment;

[0159] Output gate: determines which information in the cell state should be passed to the hidden state of the next moment or as the model output at the current moment; the activation value of the output gate is o t From the current input o k (t) and the hidden state h′ at the previous moment t-1It is calculated through a fully connected layer with a sigmoid activation function, W io , W ho , b io , b ho is the fully connected layer parameter, and then the final hidden state h′ is obtained by element-wise multiplication of the value of the cell state after the tanh function t :

[0160] o t =σW io o k (t)+b io +W ho h′ t-1 +b ho ) (32)

[0161] h′ t =o t *tanh(c′ t ) (33)

[0162] The result of the output gate is o t After passing through the fully connected layer with RELU function, the action output is obtained by sampling through Categorical function;

[0163] For the kth user, let the parameter of the target policy network be θ k , the behavior strategy network parameter is θ′ k , the network structure of the target strategy and the behavior strategy is the same, but the parameters are different. The strategy probability value generated by the target strategy network is expressed as π k (a k (t)|θ k ,o k (t)), the strategy probability value generated by the behavior strategy network is expressed as π k (a k (t)|θ k ′,o k (t)), after the user interacts with the environment, the r(t) and V generated by this step of interaction will be ζ (s(t)), a k (t),π k (a k (t)|θ k ,o k (t)) and o k (t+1) is stored in the cache (replaybuffer). When entering the network parameter update phase, the mini-batch cache data is taken out from the cache for parameter update. In the above LSTM-NOISEMAPPO algorithm, the policy objective function of each user is expressed as:

[0164]

[0165] Where ι(·) represents the observed state o of user k. k (t) obeys the distribution, V k (·) represents the state value function of the kth user, where k The gradient of is expressed as:

[0166]

[0167] In the formula, Q k (o k (t),a k (t)) indicates that user k is in the observed state o k (t), action is a k The action-value function at time (t);

[0168] It is very difficult to directly calculate formula (26), so we approximate the policy gradient by calculating an unbiased estimate of formula (35), as shown in formula (36):

[0169]

[0170] make represents the advantage function, where α and n adv are noise factors respectively, and the policy gradient is expressed as follows:

[0171]

[0172] According to the above formula (37) and introducing the idea of ​​cutting proxy function, the loss function of the policy network is expressed as follows:

[0173]

[0174] Where r(θ) = π k (a k (t)|θ k ,o k (t)) / π k (a k (t)|θ′ k ,o k (t)) is the importance weight, ε is a hyperparameter, S represents the entropy of the strategy, δ is a hyperparameter that controls the entropy coefficient, and clip(·) is the clipping function, that is:

[0175]

[0176] In the formula, x represents a variable;

[0177] The loss function of the review network is as follows:

[0178]

[0179] In the formula, The parameter is δ H Huber function, R represents the discounted reward for the user to interact with the environment until the end of a game, value clipped It is expressed as follows:

[0180]

[0181] In the formula, They represent the state value functions of the comment network output before and after the update, and ε1 is the parameter of the cutting function.

[0182] The training process of the designed LSTM-NOISEMAPPO algorithm is shown in Algorithm 1.

[0183]

[0184]

[0185] In one embodiment, a semantic communication system resource allocation system for image restoration tasks is provided, the system comprising:

[0186] The first module is used to establish a semantic communication system, including a base station and a number of users randomly distributed within the coverage of the base station signal; each user stores a neural network for extracting image semantic information, and the base station is equipped with an edge computing node, which stores a neural network for restoring images using semantic information;

[0187] A second module is used for the user to send a pilot signal to the base station, and different pilot signals are orthogonal to each other;

[0188] A third module is used for the base station to perform channel estimation on the uplink channel of each user based on the pilot signals sent by all users;

[0189] The fourth module is used to perform theoretical analysis on the SINR and the statistical characteristics of the SINR of the transmitted signal at the receiving end based on the uplink channel estimation result;

[0190] The fifth module is used to analyze the transmission energy consumption of semantic information sent by each user by using the SINR of each user, and calculate the theoretical average symbol error performance of the signal sent by each user according to the modulation mode;

[0191] The sixth module is used to obtain a mapping curve of the number of CPU cycles required for the user to extract semantic information and the base station to restore image information under different semantic extraction rates, perform numerical fitting on the mapping curve, and calculate the computational energy consumption for extracting semantic information and restoring image information based on the fitting results;

[0192] The seventh module is used to construct an optimization problem and simplify the problem to obtain the semantic communication resource allocation problem that ultimately needs to be optimized;

[0193] The eighth module is used to design and build a new deep reinforcement learning algorithm based on a multi-agent proximal strategy optimization algorithm to solve the semantic communication resource allocation problem. During the training process, a dynamically changing training environment is constructed by updating the user's position and channel status in each interaction round to train the user offline until the training results converge.

[0194] For the specific definition of the semantic communication system resource allocation system for image restoration tasks, please refer to the definition of the semantic communication system resource allocation method for image restoration tasks mentioned above, which will not be repeated here. Each module in the above-mentioned semantic communication system resource allocation system for image restoration tasks can be implemented in whole or in part through software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above-mentioned modules.

[0195] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method for allocating resources of a semantic communication system for image restoration tasks when executing the computer program.

[0196] For the specific limitations of each step, please refer to the limitations of the resource allocation method of the semantic communication system for image restoration tasks in the above text, which will not be repeated here.

[0197] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method for allocating resources of a semantic communication system for image restoration tasks are implemented.

[0198] For the specific limitations of each step, please refer to the limitations of the resource allocation method of the semantic communication system for image restoration tasks in the above text, which will not be repeated here.

[0199] As a specific example, in one of the embodiments, the present invention is further verified and explained.

[0200] The computer graphics card used in the simulation of this embodiment is configured as GeForce RTX 4090 GPU, the CPU is configured as i9-14900K, the RAM is configured as 64GB, and the software is configured as Python 3.12 and Pytorch 1.12.1. , where r k represents the actual distance from the kth user to the base station, r h represents the minimum distance between all users and the base station, v represents the path loss coefficient, satisfy Among them The modulation mode of the user's transmitted signal is 16-QAM. The amount of original image data that each user needs to transmit in each training round is obtained by referring to the image size of the MNIST dataset. Task k (t) is uniformly distributed between [6,13]KB. During the training process, the number of episodes per game is set to 100, the number of training games is set to 45000, the hidden state dimension of the user policy network is set to 64, the sequence length of the observation state input is set to 1, and the learning rate of the policy network and evaluation network parameter update is set to 5×10 -4 , the clipping range ε ​​in the clipping function clip(g) is set to 0.2, and the Huber function The parameter δ H Set to 10, the parameters a~U(0.95,0.99), b~U(1.01,1.05) in the scaling factor ε in the global state s(t), The parameter x and advantage function in The noise factors α and n in adv The setting method adopts the existing method, and the weight coefficients λ1, λ2, λ3, and λ4 in the global reward r(t) are set to 1, 0.5, 0.1, and 0.1 respectively. The other simulation parameters are shown in Table 1.

[0201] Table 1 Simulation parameters

[0202]

[0203]

[0204] In order to verify the good performance of the proposed algorithm in the simulation, this embodiment adopts three algorithms (MAPPO algorithm, NOISE-MAPPO algorithm and LSTM-MAPPO algorithm) as comparison algorithms of the proposed LSTM-NOISE MAPPO algorithm.

[0205] Figure 4 The following is a graph showing the relationship between the average energy consumption of all users in each training round and the number of training rounds under different algorithms. By observing the four simulation results, we can find three phenomena:

[0206] First, the energy consumption of the proposed algorithm and the comparison algorithm after convergence are almost in the same range. The average of the data after training convergence is taken, and the average values ​​of the four algorithms after stabilization are 12.2038mJ (proposed algorithm), 11.8784mJ (MAPPO), 11.6352mJ (LSTM MAPPO), and 12.1100mJ (NOISE MAPPO). By comparing the data, it is found that the training effect of the proposed algorithm and the comparison algorithm is slightly reduced, but the difference is not large. It can be considered that the proposed algorithm has almost the same training effect as the comparison algorithm.

[0207] Second, the MAPPO algorithm and the NOISE MAPPO algorithm are in the initial training process (episodes ≤ 10 4 ) The energy consumption value has obvious large-scale fluctuation phenomenon, while the proposed algorithm and LSTM MAPPO algorithm do not have this phenomenon during the training process. Further analysis of the training data shows that when the proposed algorithm reaches the same training result variance level as the MAPPO algorithm and the NOISE MAPPO algorithm (episodes ≥ 10 4 ) The number of training rounds can be reduced by 80% (compared with MAPPO), 66.7% (compared with LSTM MAPPO), and 80% (compared with NOISE MAPPO), respectively. The results show that the training convergence speed of the proposed algorithm is significantly faster.

[0208] Third, the variance of the training results of the entire training process was obtained, and the corresponding variance values ​​of the training results of the four algorithms were 2.4597 (proposed algorithm), 4.4169 (MAPPO), 2.8430 (LSTM MAPPO), and 3.2299 (NOISE MAPPO). This shows that the proposed algorithm is more stable than all the comparison algorithms as a whole, with stability improved by 44.31% (compared with MAPPO), 13.48% (compared with LSTM MAPPO), and 23.85% (compared with NOISE MAPPO). And in the middle and late stages of training (referring to episodes ≥ 3×10 4 ), the energy consumption of the proposed algorithm fluctuates further less than that of the comparison algorithm. The corresponding training result variance values ​​of the four algorithms are 1.0187 (proposed algorithm), 2.0465 (MAPPO), 2.1683 (LSTM MAPPO), and 2.0850 (NOISE MAPPO), indicating that the proposed algorithm can make the network energy consumption more stable in the middle and late stages of training, and the stability is further improved by 50.22% (compared with MAPPO), 53.02% (compared with LSTM MAPPO), and 51.14% (compared with NOISE MAPPO).

[0209] Figure 5The relationship between the average delay of all users in each training round and the number of training rounds under different algorithms is shown in Figure 2. It can also be seen that the proposed algorithm is faster than the comparison algorithm in the middle and late stages of training (episodes ≥ 3×10 4 ) will make the system's latency performance significantly more stable. By analyzing the latency data of the system under the four algorithms, it can be found that the latency stability brought by the proposed algorithm to the network in the middle and late stages of training can be improved by 96.6% (compared to MAPPO), 96.7% (compared to LSTM MAPPO) and 96.8% (compared to NOISE MAPPO) respectively compared with the comparison algorithms. At the same time, the latency performance of the four algorithms after training is stabilized is almost the same. In addition, the latency performance of the MAPPO algorithm and the NOISE MAPPO algorithm once again reflects the phenomenon of large-scale fluctuations in training effects. The proposed algorithm has a good performance in episodes = 10. 3 It can achieve the same performance as MAPPO algorithm and NOISE MAPPO algorithm in episodes ≥ 10. 4 This is consistent with the comparison results of energy consumption performance.

[0210] Figure 6 The relationship between the average link rate of all users in each round and the number of training rounds under different algorithms is shown in the figure. The average link rate of all users in each round under the four algorithms is 13.627Mb / s (proposed algorithm), 13.635Mb / s (MAPPO), 13.64Mb / s (LSTM MAPPO), and 13.63Mb / s (NOISEMAPPO), respectively, and the variances are 6.0294×10 11 (Proposed algorithm), 6.0134×10 11 (MAPPO), 6.0378×10 11 (LSTMMAPPO), 6.0435×10 11 The maximum and minimum values ​​are 16.953Mb / s and 10.645Mb / s (proposed algorithm), 17.32Mb / s and 10.791Mb / s (MAPPO), 17.178Mb / s and 10.881Mb / s (LSTM MAPPO), and 16.927Mb / s and 10.647Mb / s (NOISE MAPPO). The comparative analysis of the above three indicators shows that there is almost no difference between the proposed algorithm and the comparative algorithm in terms of network link performance.

[0211] Table 2 shows the mean and variance of the maximum average symbol error rate of all users per round. By comparing the data, it is found that the proposed algorithm has a small performance improvement in the mean and variance of symbol error performance compared with the comparison algorithm. The mean improvement is 4% (compared with MAPPO), 10.12% (compared with LSTM MAPPO), and 16.33% (compared with NOISE MAPPO). By comparing the variance, it is found that the stability improvement is 4.11% (compared with MAPPO), 16.48% (compared with LSTM MAPPO), and 9.75% (compared with NOISE MAPPO).

[0212] Table 2 Comparison of network symbol error performance between the proposed algorithm and the comparison algorithm

[0213]

[0214]

[0215] Through the performance analysis and comparison in the above four aspects, it can be seen that the proposed algorithm can fully reach the level of the comparison algorithm in terms of network energy consumption, delay, link rate and symbol error performance, and has significantly faster convergence speed and better convergence stability than the comparison algorithm. At the same time, it also shows a slight performance improvement in symbol error performance.

[0216] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments, and the above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A semantic communication system resource allocation method for image restoration tasks, characterized in that: The method comprises: Step 1: Establish a semantic communication system, including a base station and a number of users randomly distributed within the coverage of the base station signal; each user stores a neural network for extracting image semantic information, and an edge computing node is installed at the base station, which stores a neural network for restoring images using semantic information; Step 2, the user sends a pilot signal to the base station, and different pilot signals are orthogonal to each other; Step 3, the base station performs channel estimation on the uplink channel of each user based on the pilot signals sent by all users; Step 4, based on the uplink channel estimation result, a theoretical analysis is performed on the SINR and the statistical characteristics of the SINR of the transmitted signal at the receiving end; Step 5, using the SINR of each user to analyze the transmission energy consumption of the user sending semantic information, and calculating the theoretical average symbol error performance of the signal sent by each user according to the modulation mode; Step 6, obtaining a mapping curve of the number of CPU cycles required for the user to extract semantic information and the base station to restore image information at different semantic extraction rates, performing numerical fitting on the mapping curve, and calculating the computational energy consumption for extracting semantic information and restoring image information based on the fitting result; Step 7, construct an optimization problem, and simplify the problem to obtain the semantic communication resource allocation problem that ultimately needs to be optimized; Step 8: Design and build a new deep reinforcement learning algorithm based on the multi-agent proximal strategy optimization algorithm to solve the semantic communication resource allocation problem. During the training process, a dynamically changing training environment is constructed by updating the user's position and channel state in each interaction round to train the user offline until the training results converge.

2. The method for allocating resources of a semantic communication system for image restoration tasks according to claim 1, characterized in that: Step 1 also includes: Before executing the semantic communication task, the semantic transceiver network of the semantic communication sender and receiver is trained, and after the training, the transceiver network parameters corresponding to the specific semantic background knowledge are obtained; the semantic transceiver network includes a neural network for extracting image semantic information at the user end and a neural network at the edge computing node for restoring the image using the semantic information.

3. The method for allocating resources of a semantic communication system for image restoration tasks according to claim 1, characterized in that: In step 2, at the beginning of each channel coherence interval, M single-antenna users in the cell simultaneously send mutually orthogonal pilot signals to the base station. The number of antennas in the base station is represented by N. r , the number of pilot signals is M, and the pilot dimension size is 1×τ p , the pilot vector sent by the kth user is represented by Φ k , whose elements are represented by satisfy τ p Greater than or equal to M, the transmission power of the pilot signal is the maximum transmission power of the user, expressed as p k .

4. The method for allocating resources of a semantic communication system for image restoration tasks according to claim 3, characterized in that: Step 3 specifically includes: make N represents the number of base stations in the cell r The uplink fading between the root antenna and the kth user is expressed as: In the formula, h k YesN r ×1 vector, each element of which follows a complex Gaussian distribution with a mean of 0 and a variance of 1, indicating the small-scale fading between the user and each antenna of the base station; β k is a scalar, representing the large-scale fading between the kth user and the base station; After all users send pilot signals, the received signal at the base station is represented by Y: In the formula, the dimension of Y is N r ×M, M is the number of pilot signals, n is the dimension N r ×M complex additive Gaussian noise matrix, whose elements satisfy independent and identical distribution CN(0,σ 2 ), CN(0,σ 2 ) means the mean is 0 and the covariance is σ 2 The complex Gaussian distribution of is obtained by the orthogonality between different pilot vectors. Φ j represents the pilot signal sent by the jth user, and H represents the conjugate transpose; make Multiply both sides of the equal sign of formula (2) on the right to obtain: In the formula, The dimension size is N r ×1; Based on formula (3), the MMSE method is used to estimate the channel and the uplink channel of the kth user is obtained as 5. The method for allocating resources of a semantic communication system for image restoration tasks according to claim 4, characterized in that: Step 4 specifically includes: (1) After channel estimation, the received signal of the base station during the communication process is expressed as y: In the formula, x k represents the transmitted signal of the kth user, p′ k represents the signal transmission power of the kth user, For any k-th user, the base station side calculates the dimension of 1×N r The weight vector v k The inner product of the received signal y is used to linearly combine the received signal, which can be expressed as: Where p′ k' represents the signal transmission power of the k'th user, x k' represents the transmitted signal of the k'th user, represents the channel estimation value of the k'th user; Based on formula (6), the maximum SINR of the kth user is expressed as γ k : In the formula, The dimension is N r ×N r The identity matrix, C k′ is the mean square value of the channel estimation error of the k′th user, expressed as: In the formula, β k' is a scalar, representing the large-scale fading between the k′th user and the base station; (2) Analyze the probability density function of the uplink channel SINR of any user by asymptotic method Let matrix The eigenvalues ​​of converge to λ1. According to the Marchenko-Pastur distribution criterion, the mean of λ1 is calculated as: Therefore, formula (7) is simplified to: In the formula, represents the jth element of the channel estimation vector of the kth user, and the expression of φ is as follows: The statistical distribution of SINR in equation (10) is represented by the convolution of the two distributions on the right hand side of the equation, expressed as: In the formula, N1=M-1 N2=N r -M+1 6. The method for allocating resources of a semantic communication system for image restoration tasks according to claim 4, characterized in that: Step 5 specifically includes: For various digital modulation systems, the average symbol error rate is expressed by the Gaussian Q function: Among them, the Gaussian Q function is expressed as: Parameters a and b are constants determined by the specific modulation scheme; Substituting formula (14) into formula (13), we get: Among them, the closed expression of C(x) is as follows: in, Substituting C(x) into formula (15) we get expression.

7. The method for allocating resources of a semantic communication system for image restoration tasks according to claim 1, characterized in that: Step 6 specifically includes: For each user, the energy consumption consists of two parts: the energy consumption of extracting semantic information E 1k And the energy consumption of uplink transmission E 2k ; (1) The energy consumption E of extracting semantic information 1k for: E 1k =κy 1k (D k ,S k )f 2 (17) In the formula, y 1k (D k ,S k ) represents the number of CPU cycles for extracting semantic information of the kth user in the cell, D k and S k They represent the data size before and after semantic extraction for the kth user, and the semantic extraction rate ρ k Related, f represents the computing capacity of the kth user; κ represents the energy consumption coefficient; The numerical fitting method is used to obtain y 1k (D k ,S k ) and ρ k The relationship between them is expressed as follows: y 1k (D k ,S k )=η1ρ k +η2 (18) Where η1>0 and η2>0 are constant parameters respectively; The time it takes for a user to extract semantic information is: (2) Energy consumption E of the uplink transmission 2k for: E 2k =t 2k p′ k (20) Among them, the time required for uplink transmission is t 2k : In the formula, Cap k Cap represents the uplink channel capacity of the kth user, which is expressed as k =Wlog2(1+γ k ), where W is the uplink bandwidth allocated to each user, S k The amount of data after extracting semantic information for the kth user; (3) For edge computing nodes, the energy consumption is the computational energy consumption when performing image restoration on the semantic information sent by users in the cell, expressed as: In the formula, E 3k represents the energy consumption of decoding the information sent by the kth user, It represents the number of CPU cycles required to decode the information sent by the kth user. and They represent the amount of semantic information data after transmission through the wireless channel and the amount of data after semantic decoding. Since the neural network structure of the semantic encoding and semantic decoding process is symmetrical, it is considered that f 3k represents the computing capacity allocated to the kth user; The corresponding time consumption is: In summary, the total energy consumption E of the kth user to perform his semantic communication task is k Expressed as: E k =E 1k +E 2k +E 3k .

8. The method for allocating resources of a semantic communication system for image restoration tasks according to claim 7, characterized in that: The optimization problem constructed in step 7 is: Among them, constraint C1 is the delay constraint for each user in the cell to perform the semantic communication task, t th represents the upper limit of the delay constraint for each user, C2 represents the accuracy constraint for executing the semantic task for each user, and u k (D k ,S k ) represents the semantic accuracy of semantic communication of the kth user, A k represents the lower limit of the semantic accuracy constraint of the kth user, C3 and C4 are the computing capacity constraints of the edge computing nodes, and F max represents the maximum computing capacity of the edge computing node, C5 is the signal transmission power constraint for each user, and p level1 , p level2 , p level3 , p level4 They represent the optional power, C6 is the link symbol error performance constraint for each user, is the upper limit of the symbol error performance constraint for each user, C7 represents the sum constraint of the link rate of all users in the cell, Γ th Indicates the lower limit of the link rate sum constraint, The optimization problem P1 is equivalent to: Where C2 constraint is the constraint condition of each user’s semantic extraction rate, ρ th Represents the lower limit of the semantic extraction rate constraint for each user.

9. The method for allocating resources of a semantic communication system for image restoration tasks according to claim 8, characterized in that: Step 8 specifically includes: For the optimization problem P2, the overall solution idea is: decompose the problem into two sub-problems, and solve the two sub-problems in turn to make P2 converge to the optimal value; First, we fix the variable ρ k and p′ k When P2 is transformed into a single variable f 3k The sub-problem is expressed as follows: P3 is a convex optimization problem, which is solved using the convex optimization tool CVXPY; Considering each user as an intelligent agent in deep reinforcement learning, under the condition that information can be shared among all users, the observed state of the kth user in the tth training round is defined as o k (t)@{Task k (t),Energy(t),Max_Delay(t),Max_SER(t),Action - (t)}, where Task k (t) represents the size of the original data that the kth user needs to transmit in the tth training round, Energy(t) represents the total energy consumption of all users in the t-1th training round, Max_Delay(t) represents the maximum value of the delay experienced by all users in the t-1th training round, Max_SER(t) represents the maximum value of the average symbol error rate of all user links in the t-1th training round, and the action of the kth user is defined as a k (t)@{p k (t),ρ k (t)}, where p k (t) represents the power selection scheme of the user in the tth training round, ρ k (t) represents the semantic extraction rate selection scheme of the user in the tth training round, Action - (t) represents the set of actions taken by users other than the user himself in the t-1th training round, that is, Action - (t)={a1(t)∪a2(t)...a k-1 (t)∪a k+1 (t)...∪a M (t)}; The training adopts a centralized training method, defining the global state as the set of all user states, that is, The reward value of each user in the training round is expressed as: r(t) = -λ1Energy(t) + λ2(Γ-Γ th )+λ3(t th -Max_Delay(t))+λ4(P e th -Max_SER(t)), the 2nd, 3rd, and 4th items on the right side of the equation r(t) are penalty items, λ1, λ2, λ3, and λ4 are weight coefficients of the corresponding items, and all users share the same reward value. In each round, each user uses their own strategy network to output actions according to their own observation state and uploads their own state to the base station. The edge computing node at the base station calculates r(t) and uses the evaluation network to output the global state value function V after obtaining the global state information. ζ (s(t)), where ζ represents the parameters of the evaluation network.

10. The method for allocating resources of a semantic communication system for image restoration tasks according to claim 9, characterized in that: The new deep reinforcement learning algorithm based on the multi-agent proximal policy optimization algorithm in step 8 is an improvement on the multi-agent proximal policy optimization algorithm MAPPO, which specifically includes: (1) The set of state values ​​of all training rounds of each game is regarded as a time series. The policy network of each user is designed as a long short-term memory neural network that can extract time series features to replace the MLP network structure in the policy network of the MAPPO algorithm. The user's policy in each training round is obtained by calculating the user's observed state through the LSTM network. (2) In the evaluation function, i.e., the global state value function V ζ In the output process of (s(t)), noise is added to the global state s(t), and the global state with noise added is It is expressed as follows: In the formula, concat(·) function represents the concatenation function, which is used to horizontally concatenate s(t) and the noise matrix x(t). is the global state after adding noise, each element in x The mean is 0 and the variance is Normal distribution, ε is the scaling factor, ε:U(a,b), U(a,b) represents a uniform distribution between a and b; in addition, a noise factor is added in the calculation process of the advantage function; In the tth round of the user offline training phase, the policy network of the kth user is based on the observed state o of the user. k (t) Choose the appropriate action a k (t), the action output process is expressed as follows: Forget gate: determines which information in the cell state at the previous moment needs to be forgotten; the activation value f of the forget gate t From the current input o k (t) and the hidden state h at the previous moment t ' -1 It is calculated through a fully connected layer with a sigmoid activation function, W if , W hf 、b if 、b hf Parameters of the fully connected layer: f t =σ(W if o k (t)+b if +W hf h′ t-1 +b hf ) (28) Input gate: determines which information in the current input should be added to the cell state. It includes two parts: first, the sigmoid function is used to determine the information acceptance weight i t , and secondly, the candidate state is calculated by the tanh function, g t , W ii , W hi 、b ii 、b hi , W ig , W hg 、b ig 、b hg They are the corresponding fully connected layer parameters: i t =σ(W ii o k (t)+b ii +W hi h′ t-1 +b hi ) (29) g t =tanh(W ig o k (t)+b ig +W hg h′ t-1 +b hg ) (30) Cell state update: Combine the results of the forget gate and the input gate to update the cell state c t ': c′ t =f t *c t-1 +i t *g t (31) In the formula, * represents the bitwise multiplication operation, c t-1 Indicates the cell state at the previous moment; Output gate: determines which information in the cell state should be passed to the hidden state of the next moment or as the model output at the current moment; the activation value of the output gate is o t From the current input o k (t) and the hidden state h′ at the previous moment t-1 It is calculated through a fully connected layer with a sigmoid activation function, W io , W ho 、b io 、b ho is the fully connected layer parameter, and then it is element-wise multiplied with the value of the cell state after the tanh function to obtain the final hidden state h t ': o t =σ(W io o k (t)+b io +W ho h′ t-1 +b ho ) (32) h t =o t *tanh(c′ t ) (33) The result of the output gate is o t After passing through the fully connected layer with RELU function, the action output is obtained by sampling through Categorical function; For the kth user, let the parameter of the target policy network be θ k , the behavior strategy network parameter is θ k ′, the network structure of the target strategy and the behavior strategy is the same, but the parameters are different. The strategy probability value generated by the target strategy network is expressed as π k (a k (t)|θ k ,o k (t)), the strategy probability value generated by the behavior strategy network is expressed as π k (a k (t)|θ k ′,o k (t)), after the user interacts with the environment, the r(t) and V generated by this step of interaction will be ζ (s(t)), a k (t),π k (a k (t)|θ k ,o k (t)) and o k (t+1) is stored in the cache. When the network parameter update phase is entered, the mini-batch cache data is taken out from the cache for parameter update. In the above LSTM-NOISEMAPPO algorithm, the policy objective function of each user is expressed as: Where l(·) represents the observed state o of user k k (t) obeys the distribution, V k (·) represents the state value function of the kth user, where k The gradient of is expressed as: In the formula, Q k (o k (t),a k (t)) indicates that user k is in the observed state o k (t), action is a k The action-value function at time (t); The policy gradient is approximated by calculating an unbiased estimate of formula (35), as shown in formula (36): make represents the advantage function, where α and n adv are noise factors respectively, and the policy gradient is expressed as follows: According to the above formula (37) and the idea of ​​cutting the proxy function, the loss function of the policy network is expressed as follows: Where r(θ) = π k (a k (t)|θ k ,o k (t)) / π k (a k (t)|θ' k ,o k (t)) is the importance weight, ε is a hyperparameter, S represents the entropy of the strategy, δ is a hyperparameter that controls the entropy coefficient, and clip(·) is the clipping function, that is: In the formula, x represents a variable; The loss function of the review network is as follows: In the formula, The parameter is δ H Huber function, R represents the discounted reward for the user to interact with the environment until the end of a game, value clipped It is expressed as follows: In the formula, They represent the state value functions of the comment network output before and after the update, and ε1 is the parameter of the cutting function.

Images