Multi-task semantic communication system resource allocation method based on deep reinforcement learning

Abstract

The invention discloses a multi-task semantic communication system resource allocation method based on deep reinforcement learning, and belongs to the technical field of wireless communication, and the method comprises the steps: constructing a semantic relay assisted multi-task semantic communication network model; based on a multi-task semantic communication network model, establishing a multi-task resource allocation optimization model by taking maximization of multi-task user experience quality as a target; wherein the multi-task resource allocation optimization model is used for adjusting power allocation, sub-channel allocation and transmission semantic symbolic number allocation of each task so as to maximize the multi-task user experience quality; and constructing and training a hybrid deep reinforcement learning model to obtain a strategy network capable of realizing optimal resource allocation, and performing wireless resource allocation optimization on the basis of the multi-task resource allocation optimization model. According to the invention, while the limitation of the user equipment is met, the semantic network deployment overhead is reduced, and the efficient utilization of spectrum bandwidth resources is realized.

Description

Technical Field

[0001] The present invention relates to the field of wireless communication technology, and in particular to a resource allocation method for a multi-task semantic communication system based on deep reinforcement learning. Background Art

[0002] With the continuous surge in data-intensive applications, mobile network data traffic will surge exponentially in the future. Semantic communication can significantly reduce the amount of data transmitted by transmitting the semantic information most relevant to the task and reducing the transmission of redundant information. It has become an important communication paradigm to alleviate the shortage of spectrum resources and achieve efficient resource utilization.

[0003] At present, the information sources transmitted by semantic communication are usually text and images. The information categories, intentions, and preferences required by users in the same cellular network may have personalized deviations. In traditional semantic communication scenarios, the sender extracts the semantic features behind the source to be transmitted based on a deep learning model. This process is a black box, and the extracted semantic features are unexplainable. The sender and receiver need to establish a shared knowledge base for further processing to complete related tasks, which is very costly for the deployment of semantic communication networks.

[0004] In addition, considering the computing and storage resource limitations of existing mobile user devices, edge devices, etc., it is impossible to deploy semantic codecs based on deep learning. In order to achieve high-quality, low-latency communication in a communication environment with scarce resources and limited devices, it is urgent to design new semantic communication methods that take into account user limitations while improving the spectrum resource utilization of wireless communications.

[0005] In summary, the current semantic network has the problem of high deployment overhead and inefficient use of spectrum bandwidth resources. Therefore, it is necessary to study more efficient and personalized resource allocation methods for multi-task semantic communication systems to achieve efficient resource utilization while meeting the needs of all users as much as possible. Summary of the invention

[0006] The present invention provides a resource allocation method for a multi-task semantic communication system based on deep reinforcement learning to solve the technical problems of large deployment overhead and ineffective utilization of spectrum bandwidth resources in current semantic networks.

[0007] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0008] On the one hand, the present invention provides a multi-task semantic communication system resource allocation method based on deep reinforcement learning, and the multi-task semantic communication system resource allocation method based on deep reinforcement learning comprises:

[0009] Construct a multi-task semantic communication network model assisted by semantic relay;

[0010] Based on the multi-task semantic communication network model, a multi-task resource allocation optimization model is established with the goal of maximizing the multi-task user experience quality; wherein the multi-task resource allocation optimization model is used to adjust the power allocation, sub-channel allocation and transmission semantic symbol number allocation of each task to maximize the multi-task user experience quality;

[0011] A hybrid deep reinforcement learning model is constructed and trained to obtain a policy network that can achieve optimal resource allocation. Based on the multi-task resource allocation optimization model, wireless resource allocation optimization is performed.

[0012] Furthermore, the multi-task semantic communication network model includes: a base station, a semantic relay, and a terminal user with limited device computing and storage; wherein the terminal user includes an image user and a text user;

[0013] The base station is equipped with a scene graph generation model for generating interpretable image semantic features and a DeepSC encoder for extracting text semantic information;

[0014] The semantic relay is deployed with a DeepSC decoder to assist in the decoding and transmission of text information, and transmit the decoding result to the text user through bit encoding;

[0015] The intelligent tasks in the network include image transmission tasks and text transmission tasks; among them,

[0016] For image transmission tasks, the base station captures the objects and their relationships in the image through a scene graph generation model, obtains semantic information in an interpretable text format, and further transmits it to the image user through bit encoding;

[0017] For text transmission tasks, a semantic relay is introduced near the text user to assist in the decoding and forwarding of semantic information. The semantic relay semantically decodes the received text semantic information and transmits it to the text user through bit encoding.

[0018] Furthermore, based on the multi-task semantic communication network model, a multi-task resource allocation optimization model is established with the goal of maximizing the multi-task user experience quality, including:

[0019] Based on the multi-task semantic communication network model, obtaining the semantic transmission rate and semantic similarity of each task;

[0020] Based on the semantic transmission rate and semantic similarity of each task, a multi-task resource allocation optimization model is established with the goal of maximizing the multi-task user experience quality.

[0021] Furthermore, the semantic similarity of the image transmission task is evaluated by using the cosine of the angle between the image vector and its normalized semantic triplet vector; the semantic similarity of the text transmission task is obtained by performing data regression and curve fitting on the data of the base station and semantic relay under different transmission semantic symbol numbers and signal-to-noise ratios.

[0022] The semantic rate is defined as the rate at which users receive task information, which can be expressed as:

[0023]

[0024] in, is the semantic rate of task n; γ n is the signal-to-noise ratio of the transmission channel of task n; B is the channel bandwidth;

[0025] The signal-to-noise ratio γ of the transmission channel of task n n The calculation formula is:

[0026]

[0027] Among them, ρ m,n is the situation where task n occupies subchannel m during transmission, p n is the transmission power when task n is transmitted to the user; σ 2 is the noise power; h m,n is the channel gain of the nth task on the mth channel, including large-scale fading and small-scale Rayleigh fading considering path loss and shadow effects; M is the number of subchannels in the system.

[0028] Furthermore, the multi-tasking user experience quality is expressed as:

[0029]

[0030] Among them, QoE represents the quality of multi-task user experience; λ is the user's preference parameter for semantic rate; e is a natural number base; β is the semantic rate score parameter; ε is the semantic similarity score parameter; The minimum semantic rate requirement for task data transmission; ξ req Minimum semantic similarity requirement for task data transmission; is the semantic rate of task n; ξ n is the semantic similarity of task n, is the semantic rate score of task n, is the semantic similarity score of task n.

[0031] Furthermore, the multi-task resource allocation optimization model P0 is expressed as:

[0032]

[0033] Where N is the number of end users; is the approximate semantic entropy of text transmission; B is the channel bandwidth; N image is the number of image users; N text is the number of text users; k n is the average number of semantic symbols transmitted by task n; M is the number of subchannels in the system; P max is the maximum constraint on the transmission power of task n; G th Minimum score requirements for semantic similarity and semantic rate when transferring for task n.

[0034] Furthermore, a hybrid deep reinforcement learning model is constructed and trained, including:

[0035] The reinforcement learning algorithm is extended to discrete space and continuous space respectively, and a two-layer hybrid deep reinforcement learning model is constructed; wherein the hybrid deep reinforcement learning model takes the system channel signal-to-noise ratio, task transmission semantic rate and task transmission semantic similarity as states, and takes the sub-channel allocation scheme, transmission semantic symbol allocation scheme and transmission power allocation scheme as actions; wherein the transmission semantic symbol allocation scheme refers to the average number of transmission semantic symbols during text transmission and image transmission; the hybrid deep reinforcement learning model includes a reinforcement learning model for discrete action space and a reinforcement learning model for continuous action space; wherein the reinforcement learning model for discrete action space is used to optimize discrete action variables, including sub-channel allocation and transmission semantic symbol allocation; the reinforcement learning model for continuous action space is used to optimize continuous action variables, namely transmission power allocation;

[0036] Train the agent based on a hybrid deep reinforcement learning model to obtain a policy network that achieves optimal resource allocation.

[0037] Furthermore, the reward function r of the hybrid deep reinforcement learning model is expressed as:

[0038]

[0039] Where QoE is the quality of multi-tasking user experience; N is the number of terminal users; and are the penalties for not meeting the requirements of semantic similarity, transmission semantic rate, and semantic relay forwarding rate at the nth user; ν ξ , and ν s is a given penalty coefficient; N image is the number of image users; N text The number of text users.

[0040] On the other hand, the present invention further provides an electronic device, comprising a processor and a memory; wherein the memory stores at least one instruction, and the instruction is loaded and executed by the processor to implement the above method.

[0041] In yet another aspect, the present invention further provides a computer-readable storage medium, wherein at least one instruction is stored in the storage medium, and the instruction is loaded and executed by a processor to implement the above method.

[0042] The beneficial effects brought about by the technical solution provided by the present invention include at least:

[0043] Taking into account the advantages of deep reinforcement learning in solving complex resource management problems, the present invention introduces deep reinforcement learning technology to further improve the performance of multi-task semantic communication networks in device-constrained scenarios. A multi-task semantic communication system resource allocation method based on deep reinforcement learning is designed. The multi-task data collaborative transmission network is customized to compress and transmit task data with high quality and in a personalized manner. It can reduce the semantic network deployment overhead while meeting the user device limitations and achieve efficient utilization of spectrum bandwidth resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0045] Figure 1 It is a schematic diagram of the execution flow of a resource allocation method for a multi-task semantic communication system based on deep reinforcement learning provided by an embodiment of the present invention;

[0046] Figure 2 is a structural diagram of a multi-task semantic communication system provided by an embodiment of the present invention;

[0047] Figure 3 is a schematic diagram of a hybrid SAC model provided by an embodiment of the present invention;

[0048] Figure 4 It is a system block diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0049] In order to make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0050] First of all, it should be noted that in the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "exemplary" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "exemplarily" is intended to present the concept in a concrete way. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or it can be either of the two.

[0051] First embodiment

[0052] In order to achieve efficient management and utilization of wireless spectrum resources, this embodiment provides a resource allocation method for a multi-task semantic communication system based on deep reinforcement learning. The method can be implemented by a multi-task semantic communication system, which includes a base station, a semantic relay and N terminal users.

[0053] The execution flow of this method is as follows Figure 1 As shown, the following steps are included:

[0054] S1, construct a multi-task semantic communication network model assisted by semantic relay;

[0055] It should be noted that the multi-task semantic communication network uses a deep neural network to extract semantic information of image and text tasks, and introduces semantic relay to assist in decoding and forwarding text semantics, considering the computing and storage resource limitations of user terminal devices. Figure 2 As shown in the figure, the multi-task semantic communication network model includes a base station, a semantic relay and N terminal users; the terminal users include image users and text users; the intelligent tasks in the network include image transmission tasks and text transmission tasks; wherein, a neural network of a scene graph model for generating interpretable image semantic features and a DeepSC encoder neural network for extracting text semantic information are deployed on the base station device, and the scene graph model is used as an image semantic encoder to generate a series of semantic triplets representing image semantic features, thereby achieving interpretable image semantic transmission while reducing the occupancy of transmission channel resources, and DeepSC is used as a semantic encoder to encode text data into semantic symbols and send them to the semantic relay; a DeepSC decoder neural network is deployed on the semantic relay to assist in the decoding and transmission of text information, decode and restore the received semantic information, and transmit the decoding results to nearby text task terminal users through traditional bit encoding; by using the DeepSC model as a text semantic encoder, the transmission efficiency of text tasks is improved, and at the same time, a semantic relay is introduced for semantic decoding, which takes into account the limitations of user devices while improving the resource utilization of the wireless communication system. Due to limited computing and storage resources, end users cannot deploy semantic codecs based on deep neural networks.image image task users and N text The text task consists of users.

[0056] For image transmission tasks, the base station captures the objects and their relationships in the image through a scene graph generation model, obtains semantic information in an interpretable text format, and facilitates direct understanding and reading by users, and further transmits it to image users through traditional bit encoding. For text transmission tasks, considering the limitation that user devices cannot deploy semantic decoders driven by deep neural networks, a semantic relay is introduced near the text task user to assist in the decoding and forwarding of semantic information. The semantic relay deploys a resource-intensive text semantic decoder based on deep learning, performs semantic decoding on the received text semantic information, and transmits it to the text user through traditional bit encoding.

[0057] S2, based on the multi-task semantic communication network model, with the goal of maximizing the multi-task user experience quality, establish a multi-task resource allocation optimization model; wherein the multi-task resource allocation optimization model is used to adjust the power allocation, sub-channel allocation and transmission semantic symbol number allocation of each task to maximize the multi-task user experience quality;

[0058] In a feasible implementation manner, the implementation of the above S2 may include the following steps:

[0059] S21, based on the multi-task semantic communication network model, obtain the semantic transmission rate and semantic similarity of each task;

[0060] In a feasible implementation, the transmission semantic rate and semantic similarity function of each task are obtained, specifically: the semantic similarity of the image transmission task is evaluated by the cosine of the angle between the image vector and its normalized semantic triple vector. The semantic similarity is mainly determined by the number of transmission semantic triplets. The relationship between the similarity and the number of transmission semantic triplets is obtained by calculating different numbers of transmission semantic triplets for testing. The semantic similarity of the text transmission task is obtained by data regression and curve fitting of the DeepSC transceiver of the training base station and semantic relay under different transmission semantic symbol numbers and signal-to-noise ratios.

[0061] The semantic rate is defined as the rate at which users receive task information, which can be expressed as:

[0062]

[0063] in, is the semantic rate of task n; γ n is the signal-to-noise ratio of the transmission channel of task n; B is the channel bandwidth;

[0064] The signal-to-noise ratio γ of the transmission channel of task n n The calculation formula is:

[0065]

[0066] Among them, ρ m,n is the situation where task n occupies subchannel m during transmission, p n is the transmission power when task n is transmitted to the user; σ 2 is the noise power; h m,n is the channel gain of the nth task on the mth channel, including large-scale fading and small-scale Rayleigh fading considering path loss and shadow effects; M is the number of subchannels in the system.

[0067] Considering the semantic relay assisted text transmission task, the semantic rate of the base station transmitting text semantic symbols to the semantic relay is:

[0068]

[0069] Where k represents the average number of semantic symbols transmitted using the DeepSC model, represents the approximate semantic entropy of text transmission based on the DeepSC model, and B represents the channel bandwidth.

[0070] S22, based on the semantic transmission rate and semantic similarity of each task, a multi-task resource allocation optimization model is established with the goal of maximizing the multi-task user experience quality;

[0071] It should be noted that the system goal is to adjust the power allocation, subchannel allocation, and transmission semantic symbol number allocation of each task to maximize the experience quality of all users in the network. The experience quality is expressed as:

[0072]

[0073] Among them, QoE represents the quality of multi-task user experience; λ is the user's preference parameter for semantic rate; e is a natural number base; β is the semantic rate score parameter; ε is the semantic similarity score parameter; The minimum semantic rate requirement for task data transmission; ξ req Minimum semantic similarity requirement for task data transmission;ξ n is the semantic similarity of task n, is the semantic rate score of task n, is the semantic similarity score of task n.

[0074] Furthermore, the multi-task resource allocation optimization model P0 is expressed as:

[0075]

[0076] Among them, C1 represents the situation where user n occupies channel m, C2 represents that each channel can only be occupied by one user when performing task transmission, C3 represents that only one channel is occupied when transmitting tasks to each user, C4 represents the power limit when transmitting signals to users, C5 and C6 represent the minimum requirement constraints for semantic rate and semantic similarity respectively, and C7 represents that when transmitting text, semantic relay can only forward the information received from BS to users. n is the average number of semantic symbols transmitted by task n; M is the number of subchannels in the system; P max is the maximum constraint on the transmission power of task n; G th Minimum score requirements for semantic similarity and semantic rate when transferring for task n.

[0077] S3, constructing and training a hybrid deep reinforcement learning model to obtain a policy network that can achieve optimal resource allocation, and performing wireless resource allocation optimization based on the multi-task resource allocation optimization model;

[0078] It should be noted that, in this embodiment, the system channel signal-to-noise ratio γ, the task data transmission rate And the task transmission semantic similarity ξ are taken as the state, the sub-channel allocation scheme ρ, the transmission semantic symbol allocation scheme (the average number of transmission semantic symbols k when transmitting text and image tasks) and the transmission power allocation scheme p are taken as actions. Since the transmission power is a continuous variable, the channel allocation vector and the number of transmission semantic symbols are discrete variables, the reinforcement learning algorithm soft actor critic (SAC) is extended to discrete space and continuous space respectively, and a two-layer hybrid SAC algorithm is constructed to train the intelligent agent and obtain the optimal resource allocation strategy network.

[0079] Specifically, in a feasible implementation manner, the implementation of the above S3 may include the following steps:

[0080] S31, define the state space, action space and reward function of the hybrid deep reinforcement learning model;

[0081] The state space of the system is in, represents the channel signal-to-noise ratio of the system user transmission task and the signal-to-noise ratio from BS to semantic relay, Indicates the transmission rate of the system user transmission task, Indicates the semantic similarity of tasks transferred by system users.

[0082] The action space consists of discrete actions a d = {ρ, k} and continuous action a c ={p}, where ρ = [ρ1,…,ρ M ] T is the subchannel allocation matrix, ρ m=[ρ m,1 ,…,ρ m,N ] is the task allocation channel action, the sub-channel allocation vector dimension is M×N, and p is the task transmission power allocation matrix.

[0083] Based on the goals and constraints, an effective reward function is designed to constrain the behavior of the agent by combining the requirements of semantic transmission efficiency and semantic similarity. Therefore, the reward function is designed as:

[0084]

[0085] Among them, QoE is the quality of experience of the transmission task; N is the number of terminal users; and are the penalties for not meeting the requirements of semantic similarity, transmission semantic rate, and semantic relay forwarding rate at the nth user; ν ξ , and ν s is a given penalty coefficient; N image is the number of image users; N text is the number of text users. S32, build a two-layer hybrid SAC algorithm model, train the agent, and obtain the optimal resource allocation strategy network;

[0086] Among them, Figure 3 As shown in the figure, the two-layer hybrid SAC algorithm model includes a SAC model in a discrete action space and a SAC model in a continuous action space. The discrete SAC model is used to optimize discrete action variables, including subchannel allocation and the average number of transmitted semantic features, and the continuous SAC model is used to optimize continuous action variables, namely, the transmission power. The SAC algorithm introduces maximum entropy to encourage strategy exploration, enhance the search ability of the intelligent agent, and avoid falling into the local optimum. Figure 3 As shown, each SAC algorithm model contains 5 neural networks, Actor network, Critic-Q network 1, Critic-Q network 2, Target-Q network 1 and Target-Q network 2.

[0087] The training process of the two-layer hybrid SAC algorithm is as follows:

[0088] Initialize network parameters and clear the experience replay pool;

[0089] Initialize actions according to random strategies, and generate random initialization states according to initial actions;

[0090] The current state Input discrete policy network π θ (·) and continuous action policy network The Actor network of the hybrid SAC algorithm model is based on the current state Get discrete actions and continuous action Agent performs actions Status transferred to Calculate the instant reward r t , will (s t ,a t ,r t ,s t+1 ) is stored in the experience replay pool;

[0091] When the experience replay pool is larger than the storage capacity, Z sequences are sampled from it, the gradient of the network loss function (Q value) is calculated based on the samples, and the network parameters are updated, including: updating the discrete SAC framework strategy network parameters, value network parameters and entropy weight parameters; updating the continuous SAC framework strategy network parameters, value network parameters and entropy weight parameters.

[0092] Among them, the Actor loss function in the discrete SAC layer is for:

[0093]

[0094] Loss function of the Critic-Q network in the discrete SAC layer for:

[0095]

[0096] in, θ is the parameter of the Actor network in the discrete SAC layer, ψ j is the parameter of the jth Critic-Q network in the discrete SAC layer, ψ j ′ is the parameter of the jth Target-Q network in the discrete SAC layer, γ is the discount factor, and D is the experience replay pool.

[0097] The entropy weight α loss function in the discrete SAC layer is:

[0098]

[0099] Among them, H0 is the target entropy, which is a given hyperparameter.

[0100] Actor loss function in continuous SAC layer for:

[0101]

[0102] Loss function J of the Critic-Q network in the continuous SAC layer Q (ψ j )for:

[0103]

[0104] in, are the parameters of the Actor network in the continuous SAC layer, is the parameter of the jth Critic-Q network in the continuous SAC layer, are the parameters of the jth Target-Q network in the continuous SAC layer.

[0105] Entropy weights in continuous SAC layers The loss function is:

[0106]

[0107] Then, each iteration, the target policy network and target action network parameters of the hybrid SAC architecture are soft updated:

[0108] ψ′1=τψ1+(1-τ)ψ′1

[0109] ψ′2=τψ2+(1-τ)ψ′2

[0110]

[0111] When the number of iterations reaches a preset threshold, offline training is completed and a policy network that can achieve optimal resource allocation is obtained. This policy network can be used to realize resource allocation in a multi-task semantic communication system.

[0112] In summary, this embodiment takes into account the advantages of deep reinforcement learning in solving complex resource management problems. In order to improve the performance of multi-task semantic communication networks in device-constrained scenarios, deep reinforcement learning technology is introduced. A multi-task semantic communication system resource allocation method based on deep reinforcement learning is designed. Customized services are provided for the multi-task data collaborative transmission network, and task data is compressed and transmitted in a personalized manner with high quality. This can reduce the semantic network deployment overhead while meeting the user device limitations, and achieve efficient utilization of spectrum bandwidth resources.

[0113] Second embodiment

[0114] This embodiment provides an electronic device, such as Figure 4 As shown, the electronic device includes: a processor and a memory; wherein the processor and the memory can be connected via a communication bus; the memory stores at least one instruction, and the instruction is loaded and executed by the processor to implement the method of the first embodiment. In addition, the electronic device may also include a transceiver, the processor and the transceiver can be connected via a communication bus, and the transceiver is used to communicate with other devices.

[0115] Next, combine Figure 4The following is a detailed introduction to the various components of the electronic device:

[0116] Among them, the processor is the control center of the electronic device, and the electronic device may include multiple processors, each of which may be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). The processor here may be a processor or a general term for multiple processing elements. For example, the processor is one or more central processing units (CPUs), or other general-purpose processors, application specific integrated circuits (ASICs), or one or more integrated circuits configured to implement an embodiment of the present invention, such as one or more microprocessors (digital signal processors, DSPs), or one or more field programmable gate arrays (field programmable gate arrays, FPGAs), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The processor may execute various functions of the electronic device by running or executing software programs stored in the memory and calling data stored in the memory.

[0117] In a specific implementation, as an embodiment, the processor may include one or more CPUs, such as Figure 4 The CPU0 and CPU1 shown in the figure are, of course, only exemplary.

[0118] The memory is used to store the software program for executing the solution of the present invention, and the execution is controlled by the processor. The specific implementation method can refer to the above method embodiment and will not be repeated here.

[0119] Optionally, the memory may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory may be integrated with the processor or exist independently and accessed through the interface circuit ( Figure 4 (not shown) is coupled to the processor, which is not specifically limited in this embodiment of the present invention.

[0120] The transceiver may include a receiver and a transmitter ( Figure 4 The receiver is used to implement the receiving function, and the transmitter is used to implement the sending function. The transceiver can be integrated with the processor or exist independently and communicate with the electronic device through the interface circuit ( Figure 4 (not shown) is coupled to the processor, which is not specifically limited in this embodiment of the present invention.

[0121] In addition, it should be noted that Figure 4 The structure of the electronic device shown in the figure does not constitute a limitation on the device, and the actual device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. In addition, the technical effects achieved by the electronic device when executing the method of the first embodiment above can refer to the technical effects described in the first embodiment above, so they are not repeated here.

[0122] Third embodiment

[0123] This embodiment provides a computer-readable storage medium, which stores at least one instruction, and the instruction is loaded and executed by a processor to implement the method of the first embodiment. The computer-readable storage medium may be a ROM, a random access memory, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc. The instructions stored therein may be loaded by a processor in a terminal to execute the method.

[0124] In addition, it should be noted that the present invention can be provided as a method, an apparatus or a computer program product. Therefore, the embodiment of the present invention can be in the form of a full or partial hardware embodiment, a full or partial software embodiment or an embodiment combining software and hardware. Moreover, when implemented using software, the embodiment of the present invention can be in the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program codes. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions can be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center containing one or more available media sets. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a DVD), or a semiconductor medium. The semiconductor medium may be a solid state hard disk.

[0125] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0126] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device, so that a series of operation steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable terminal device provide for implementing the process in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0127] It should also be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. The terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of more restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements. In addition, the term "and / or" is only an association relationship describing the associated objects, indicating that there can be three relationships, for example, A and / or B, which can represent: A exists alone, A and B exist at the same time, and B exists alone, wherein A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding. "At least one" means one or more, and "plurality" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can be represented by: a, b, c, ab, ac, bc or abc, where a, b, c can be single or plural.

[0128] In addition, it can be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0129] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0130] In several embodiments provided by the present invention, it should be understood that the disclosed equipment, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of functional modules / units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point, the coupling or direct coupling or communication connection between each other shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms. The unit described as a separate component may or may not be physically separated, and the component displayed as a unit may or may not be a physical unit, that is, it may be located in one place, or it may be distributed on multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, each functional unit in each embodiment of the present invention can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit.

[0131] If the method is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0132] Finally, it should be noted that the above is only a preferred embodiment of the present invention. It should be pointed out that although the preferred embodiment of the present invention has been described, for ordinary technicians in this technical field, once the basic creative concept of the present invention is known, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the protection scope of the present invention. Therefore, the attached claims are intended to be interpreted as including the preferred embodiment and all changes and modifications that fall within the scope of the embodiments of the present invention.

Claims

1. A resource allocation method for a multi-task semantic communication system based on deep reinforcement learning, characterized in that: The multi-task semantic communication system resource allocation method based on deep reinforcement learning includes: Construct a multi-task semantic communication network model assisted by semantic relay; Based on the multi-task semantic communication network model, a multi-task resource allocation optimization model is established with the goal of maximizing the multi-task user experience quality; wherein the multi-task resource allocation optimization model is used to adjust the power allocation, sub-channel allocation and transmission semantic symbol number allocation of each task to maximize the multi-task user experience quality; A hybrid deep reinforcement learning model is constructed and trained to obtain a policy network that can achieve optimal resource allocation. Based on the multi-task resource allocation optimization model, wireless resource allocation optimization is performed.

2. The method for allocating resources in a multi-task semantic communication system based on deep reinforcement learning according to claim 1, characterized in that: The multi-task semantic communication network model includes: a base station, a semantic relay, and a terminal user with limited device computing and storage; wherein the terminal user includes an image user and a text user; The base station is equipped with a scene graph generation model for generating interpretable image semantic features and a DeepSC encoder for extracting text semantic information; The semantic relay is deployed with a DeepSC decoder to assist in the decoding and transmission of text information, and transmit the decoding result to the text user through bit encoding; The intelligent tasks in the network include image transmission tasks and text transmission tasks; among them, For image transmission tasks, the base station captures the objects and their relationships in the image through a scene graph generation model, obtains semantic information in an interpretable text format, and further transmits it to the image user through bit encoding; For text transmission tasks, a semantic relay is introduced near the text user to assist in the decoding and forwarding of semantic information. The semantic relay semantically decodes the received text semantic information and transmits it to the text user through bit encoding.

3. The method for allocating resources in a multi-task semantic communication system based on deep reinforcement learning as claimed in claim 2, characterized in that: Based on the multi-task semantic communication network model, a multi-task resource allocation optimization model is established with the goal of maximizing the multi-task user experience quality, including: Based on the multi-task semantic communication network model, obtaining the semantic transmission rate and semantic similarity of each task; Based on the semantic transmission rate and semantic similarity of each task, a multi-task resource allocation optimization model is established with the goal of maximizing the multi-task user experience quality.

4. The method for allocating resources in a multi-task semantic communication system based on deep reinforcement learning as claimed in claim 3, characterized in that: The semantic similarity of the image transmission task is evaluated by the cosine of the angle between the image vector and its normalized semantic triple vector; the semantic similarity of the text transmission task is obtained by data regression and curve fitting of the data of the base station and semantic relay under different transmission semantic symbol numbers and signal-to-noise ratios; The semantic rate is defined as the rate at which users receive task information, which can be expressed as: in, is the semantic rate of task n; γ n is the signal-to-noise ratio of the transmission channel of task n; B is the channel bandwidth; The signal-to-noise ratio γ of the transmission channel of task n n The calculation formula is: Among them, ρ m,n is the situation where task n occupies subchannel m during transmission, p n is the transmission power when task n is transmitted to the user; σ 2 is the noise power; h m,n is the channel gain of the nth task on the mth channel, including large-scale fading and small-scale Rayleigh fading considering path loss and shadow effects; M is the number of subchannels in the system.

5. The method for allocating resources in a multi-task semantic communication system based on deep reinforcement learning as claimed in claim 4, characterized in that: The multitasking user experience quality is expressed as: Among them, QoE represents the quality of multi-task user experience; λ is the user's preference parameter for semantic rate; e is a natural number base; β is the semantic rate score parameter; ε is the semantic similarity score parameter; The minimum semantic rate requirement for task data transmission; ξ req Minimum semantic similarity requirement for task data transmission; is the semantic rate of task n; ξ n is the semantic similarity of task n, is the semantic rate score of task n, is the semantic similarity score of task n.

6. The method for allocating resources in a multi-task semantic communication system based on deep reinforcement learning as claimed in claim 5, characterized in that: The multi-task resource allocation optimization model P0 is expressed as: Where N is the number of end users; is the approximate semantic entropy of text transmission; B is the channel bandwidth; N image is the number of image users; N text is the number of text users; k n is the average number of semantic symbols transmitted by task n; M is the number of subchannels in the system; P max is the maximum constraint on the transmission power of task n; G th Minimum score requirements for semantic similarity and semantic rate when transferring for task n.

7. The method for allocating resources in a multi-task semantic communication system based on deep reinforcement learning as claimed in claim 3, characterized in that: Build and train a hybrid deep reinforcement learning model, including: The reinforcement learning algorithm is extended to discrete space and continuous space respectively, and a two-layer hybrid deep reinforcement learning model is constructed; wherein the hybrid deep reinforcement learning model takes the system channel signal-to-noise ratio, task transmission semantic rate and task transmission semantic similarity as states, and takes the sub-channel allocation scheme, transmission semantic symbol allocation scheme and transmission power allocation scheme as actions; wherein the transmission semantic symbol allocation scheme refers to the average number of transmission semantic symbols during text transmission and image transmission; the hybrid deep reinforcement learning model includes a reinforcement learning model for discrete action space and a reinforcement learning model for continuous action space; wherein the reinforcement learning model for discrete action space is used to optimize discrete action variables, including sub-channel allocation and transmission semantic symbol allocation; the reinforcement learning model for continuous action space is used to optimize continuous action variables, namely transmission power allocation; Train the agent based on a hybrid deep reinforcement learning model to obtain a policy network that achieves optimal resource allocation.

8. The method for allocating resources in a multi-task semantic communication system based on deep reinforcement learning as claimed in claim 7, characterized in that: The reward function r of the hybrid deep reinforcement learning model is expressed as: Where QoE is the quality of multi-tasking user experience; N is the number of terminal users; and are the penalties for not meeting the requirements of semantic similarity, semantic transmission rate, and semantic relay forwarding rate at the nth user; v ξ , and v s is a given penalty coefficient; N image is the number of image users; N text The number of text users.

Images