Method for resource allocation in multi-task semantic communication system based on deep reinforcement learning

Abstract

The application discloses a multi-task semantic communication system resource allocation method based on deep reinforcement learning, and belongs to the technical field of wireless communication. The method comprises the following steps: constructing a multi-task semantic communication network model assisted by semantic relays; based on the multi-task semantic communication network model, a multi-task resource allocation optimization model is established with the target of maximizing multi-task user experience quality; wherein the multi-task resource allocation optimization model is used for adjusting power allocation, sub-channel allocation and transmission semantic symbol number allocation of each task to maximize multi-task user experience quality; a hybrid deep reinforcement learning model is constructed and trained to obtain a strategy network capable of realizing optimal resource allocation, and wireless resource allocation optimization is performed on the basis of the multi-task resource allocation optimization model. The application can meet user equipment restrictions while reducing semantic network deployment overhead and realizing efficient utilization of spectrum bandwidth resources.

Description

Technical Field

[0001] The present invention relates to the field of wireless communication technologies, and in particular to a resource allocation method for a multi-task semantic communication system based on deep reinforcement learning. Background Art

[0002] With the continuous surge in data-intensive applications, mobile network data traffic will surge exponentially in the future. Semantic communication can significantly reduce the amount of data transmitted by transmitting the semantic information most relevant to the task and reducing the transmission of redundant information. It has become an important communication paradigm to alleviate the shortage of spectrum resources and achieve efficient resource utilization.

[0003] Currently, semantic communication typically involves text and images as the information source. Users on the same cellular network may have varying degrees of individualized differences in information categories, intent, and preferences. In traditional semantic communication scenarios, the sender extracts semantic features from the source information using deep learning models. This process is a black box, and the extracted semantic features are uninterpretable. This requires the sender and receiver to establish a shared knowledge base for further processing to complete the relevant tasks, which incurs significant deployment overhead for semantic communication networks.

[0004] Furthermore, given the computational and storage resource limitations of existing mobile user devices and edge devices, it is not possible to deploy semantic codecs based on deep learning. To achieve high-quality, low-latency communication in resource-scarce and device-constrained communication environments, it is urgent to design new semantic communication methods that consider user limitations while improving spectrum resource utilization in wireless communications.

[0005] In summary, current semantic networks suffer from high deployment overhead and inefficient use of spectrum bandwidth resources. Therefore, it is necessary to research more efficient and personalized resource allocation methods for multi-task semantic communication systems, so as to achieve efficient resource utilization while meeting the needs of all users. Summary of the Invention

[0006] The present invention provides a resource allocation method for a multi-task semantic communication system based on deep reinforcement learning to solve the technical problems of large deployment overhead and inefficient utilization of spectrum bandwidth resources in current semantic networks.

[0007] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0008] In one aspect, the present invention provides a method for allocating resources in a multi-task semantic communication system based on deep reinforcement learning, the method comprising:

[0009] Construct a multi-task semantic communication network model assisted by semantic relay;

[0010] Based on the multi-task semantic communication network model, a multi-task resource allocation optimization model is established with the goal of maximizing the multi-task user experience quality; wherein the multi-task resource allocation optimization model is used to adjust the power allocation, sub-channel allocation, and transmission semantic symbol number allocation of each task to maximize the multi-task user experience quality;

[0011] A hybrid deep reinforcement learning model is constructed and trained to obtain a policy network that can achieve optimal resource allocation. Based on the multi-task resource allocation optimization model, wireless resource allocation optimization is performed.

[0012] Furthermore, the multi-task semantic communication network model includes: a base station, a semantic relay, and terminal users with limited device computing and storage; wherein the terminal users include image users and text users;

[0013] The base station is equipped with a scene graph generation model for generating interpretable image semantic features and a DeepSC encoder for extracting text semantic information;

[0014] The DeepSC decoder is deployed on the semantic relay to assist in the decoding and transmission of text information, and transmits the decoding results to the text user through bit encoding;

[0015] The intelligent tasks in the network include image transmission tasks and text transmission tasks; among them,

[0016] For image transmission tasks, the base station captures the objects and their relationships in the image through a scene graph generation model, obtains semantic information in an interpretable text format, and further transmits it to the image user through bit encoding;

[0017] For text transmission tasks, a semantic relay is introduced near the text user to assist in the decoding and forwarding of semantic information. The semantic relay semantically decodes the received text semantic information and transmits it to the text user through bit encoding.

[0018] Furthermore, based on the multi-task semantic communication network model, a multi-task resource allocation optimization model is established with the goal of maximizing the multi-task user experience quality, including:

[0019] Based on the multi-task semantic communication network model, obtaining the semantic transmission rate and semantic similarity of each task;

[0020] Based on the semantic transmission rate and semantic similarity of each task, a multi-task resource allocation optimization model is established with the goal of maximizing the multi-task user experience quality.

[0021] Furthermore, the semantic similarity of the image transmission task is evaluated by the cosine of the angle between the image vector and its normalized semantic triplet vector; the semantic similarity of the text transmission task is obtained by performing data regression and curve fitting on the data of the base station and semantic relay under different transmission semantic symbol numbers and signal-to-noise ratios.

[0022] The semantic rate is defined as the rate at which users receive task information and is expressed as:

[0023]

[0024] in, is the semantic rate of task n; γ n is the signal-to-noise ratio of the transmission channel for task n; B is the channel bandwidth;

[0025] The signal-to-noise ratio γ of the transmission channel of task n n The calculation formula is:

[0026]

[0027] Among them, ρ m,n When task n occupies subchannel m during transmission, p n is the transmission power when task n is transmitted to the user; σ 2 is the noise power; h m,n is the channel gain of the nth task on the mth channel, including large-scale fading and small-scale Rayleigh fading considering path loss and shadowing effects; M is the number of subchannels in the system.

[0028] Furthermore, the multi-tasking user experience quality is expressed as:

[0029]

[0030] Among them, QoE represents the quality of multi-task user experience; λ is the user's preference parameter for semantic rate; e is a natural number base; β is the semantic rate score parameter; ε is the semantic similarity score parameter; The minimum semantic rate requirement for task data transmission; ξ req Minimum semantic similarity requirement for task data transmission; is the semantic rate of task n; ξ n is the semantic similarity of task n, is the semantic rate score of task n, is the semantic similarity score of task n.

[0031] Furthermore, the multi-task resource allocation optimization model P0 is expressed as:

[0032]

[0033] Where N is the number of end users; is the approximate semantic entropy of text transmission; B is the channel bandwidth; N image is the number of image users; N text is the number of text users; k n is the average number of semantic symbols transmitted for task n; M is the number of subchannels in the system; P max is the maximum constraint on the transmission power of task n; G th The minimum score requirements for semantic similarity and semantic rate when transferring for task n.

[0034] Furthermore, a hybrid deep reinforcement learning model is built and trained, including:

[0035] The reinforcement learning algorithm is extended to discrete space and continuous space respectively, and a two-layer hybrid deep reinforcement learning model is constructed; wherein, the hybrid deep reinforcement learning model takes the system channel signal-to-noise ratio, task transmission semantic rate and task transmission semantic similarity as states, and takes the sub-channel allocation scheme, transmission semantic symbol allocation scheme and transmission power allocation scheme as actions; wherein, the transmission semantic symbol allocation scheme refers to the average number of transmission semantic symbols during text transmission and image transmission; the hybrid deep reinforcement learning model includes a reinforcement learning model for discrete action space and a reinforcement learning model for continuous action space; wherein, the reinforcement learning model for discrete action space is used to optimize discrete action variables, including sub-channel allocation and transmission semantic symbol allocation; the reinforcement learning model for continuous action space is used to optimize continuous action variables, namely transmission power allocation;

[0036] Train intelligent agents based on a hybrid deep reinforcement learning model to obtain a policy network that achieves optimal resource allocation.

[0037] Furthermore, the reward function r of the hybrid deep reinforcement learning model is expressed as:

[0038]

[0039] Where QoE is the quality of multi-tasking user experience; N is the number of end users; and are the penalties for not meeting the requirements of semantic similarity, semantic transmission rate, and semantic relay forwarding rate at the nth user; ν ξ 、 and ν s is a given penalty coefficient; N image is the number of image users; N text The number of text users.

[0040] On the other hand, the present invention further provides an electronic device, comprising a processor and a memory; wherein the memory stores at least one instruction, and the instruction is loaded and executed by the processor to implement the above method.

[0041] In yet another aspect, the present invention further provides a computer-readable storage medium, wherein the storage medium stores at least one instruction, and the instruction is loaded and executed by a processor to implement the above method.

[0042] The beneficial effects brought about by the technical solution provided by the present invention include at least:

[0043] Taking into account the advantages of deep reinforcement learning in solving complex resource management problems, this paper introduces deep reinforcement learning technology to further improve the performance of multi-task semantic communication networks in device-constrained scenarios. A multi-task semantic communication system resource allocation method based on deep reinforcement learning is designed. It provides customized services for the multi-task data collaborative transmission network, compresses and transmits task data with high quality and in a personalized manner, which can reduce the semantic network deployment overhead while meeting user device limitations and achieve efficient utilization of spectrum bandwidth resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0045] Figure 1 1 is a schematic diagram of the execution flow of a resource allocation method for a multi-task semantic communication system based on deep reinforcement learning provided by an embodiment of the present invention;

[0046] Figure 2 is a structural diagram of a multi-task semantic communication system provided by an embodiment of the present invention;

[0047] Figure 3 Schematic diagram of a hybrid SAC model provided by an embodiment of the present invention;

[0048] Figure 4 This is a system block diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0049] To make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be described in further detail below with reference to the accompanying drawings.

[0050] First, it should be noted that in the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as an "example" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "exemplarily" is intended to present concepts in a concrete manner. In addition, in the embodiments of the present invention, the meaning of "and / or" can be both or either of the two.

[0051] First embodiment

[0052] In order to achieve efficient management and utilization of wireless spectrum resources, this embodiment provides a resource allocation method for a multi-task semantic communication system based on deep reinforcement learning. The method can be implemented by a multi-task semantic communication system, which includes a base station, a semantic relay and N terminal users.

[0053] The execution process of this method is as follows Figure 1 As shown, the following steps are included:

[0054] S1, construct a multi-task semantic communication network model assisted by semantic relay;

[0055] It should be noted that the multi-task semantic communication network uses deep neural networks to extract semantic information from images and text tasks, and considering the computing and storage resource limitations of user terminal devices, a semantic relay is introduced to assist in the decoding and forwarding of text semantics. Specifically, Figure 2 As shown, the multi-task semantic communication network model includes a base station, a semantic relay and N terminal users; the terminal users include image users and text users; the intelligent tasks in the network include image transmission tasks and text transmission tasks; among them, a neural network of a scene graph model for generating interpretable image semantic features and a DeepSC encoder neural network for extracting text semantic information are deployed on the base station device. The scene graph model is used as an image semantic encoder to generate a series of semantic triplets representing image semantic features, thereby achieving interpretable image semantic transmission while reducing the occupancy of transmission channel resources. DeepSC is used as a semantic encoder to encode text data into semantic symbols and send them to the semantic relay; a DeepSC decoder neural network is deployed on the semantic relay to assist in the decoding and transmission of text information, decode and recover the received semantic information, and transmit the decoding results to nearby text task terminal users through traditional bit encoding; by using the DeepSC model as a text semantic encoder, the transmission efficiency of the text task is improved, and at the same time, a semantic relay is introduced for semantic decoding, which takes into account the limitations of user devices while improving the resource utilization of the wireless communication system. End users are unable to deploy semantic codecs based on deep neural networks due to limited computing and storage resources.image image task users and N text The text task consists of users.

[0056] For image transmission tasks, the base station captures the objects and their relationships in the image through a scene graph generation model, obtaining semantic information in an interpretable text format for direct user understanding and reading, which is then transmitted to the image user via traditional bit encoding. For text transmission tasks, to address the limitations of user devices in being unable to deploy deep neural network-driven semantic decoders, a semantic relay is introduced near the text task user to assist in the decoding and forwarding of semantic information. The semantic relay deploys a resource-intensive text semantic decoder based on deep learning, performs semantic decoding on the received text, and transmits it to the text user via traditional bit encoding.

[0057] S2, based on the multi-task semantic communication network model, establishes a multi-task resource allocation optimization model with the goal of maximizing the multi-task user experience quality. The multi-task resource allocation optimization model is used to adjust the power allocation, sub-channel allocation, and transmission semantic symbol number allocation of each task to maximize the multi-task user experience quality.

[0058] In a feasible implementation manner, the implementation of the above S2 may include the following steps:

[0059] S21, based on the multi-task semantic communication network model, obtains the semantic transmission rate and semantic similarity of each task;

[0060] In one feasible implementation, the semantic transmission rate and semantic similarity function for each task are obtained. Specifically, the semantic similarity for the image transmission task is evaluated using the cosine of the angle between the image vector and its normalized semantic triple vector. Semantic similarity is primarily determined by the number of transmitted semantic triplets. The relationship between similarity and the number of transmitted semantic triplets is determined by calculating different numbers of transmitted semantic triplets for testing. Semantic similarity for the text transmission task is obtained through data regression and curve fitting by training the DeepSC transceivers of the base station and semantic relay under different numbers of transmitted semantic symbols and signal-to-noise ratios.

[0061] The semantic rate is defined as the rate at which users receive task information and is expressed as:

[0062]

[0063] in, is the semantic rate of task n; γ n is the signal-to-noise ratio of the transmission channel for task n; B is the channel bandwidth;

[0064] The signal-to-noise ratio γ of the transmission channel of task n n The calculation formula is:

[0065]

[0066] Among them, ρ m,n When task n occupies subchannel m during transmission, p n is the transmission power when task n is transmitted to the user; σ 2 is the noise power; h m,n is the channel gain of the nth task on the mth channel, including large-scale fading and small-scale Rayleigh fading considering path loss and shadowing effects; M is the number of subchannels in the system.

[0067] Considering the semantic relay-assisted text transmission task, the semantic rate of the base station transmitting text semantic symbols to the semantic relay is:

[0068]

[0069] Where k represents the average number of semantic symbols transmitted using the DeepSC model, represents the approximate semantic entropy of text transmission based on the DeepSC model, and B represents the channel bandwidth.

[0070] S22, based on the semantic transmission rate and semantic similarity of each task, with the goal of maximizing the multi-task user experience quality, establish a multi-task resource allocation optimization model;

[0071] It should be noted that the system goal is to adjust the power allocation, subchannel allocation, and transmission semantic symbol number allocation of each task to maximize the experience quality of all users in the network. The experience quality is expressed as:

[0072]

[0073] Among them, QoE represents the quality of multi-task user experience; λ is the user's preference parameter for semantic rate; e is a natural number base; β is the semantic rate score parameter; ε is the semantic similarity score parameter; The minimum semantic rate requirement for task data transmission; ξ req Minimum semantic similarity requirement for task data transmission; ξ n is the semantic similarity of task n, is the semantic rate score of task n, is the semantic similarity score of task n.

[0074] Furthermore, the multi-task resource allocation optimization model P0 is expressed as:

[0075]

[0076] Among them, C1 represents the situation where user n occupies channel m, C2 represents that each channel can only be occupied by one user when performing task transmission, C3 represents that only one channel is occupied when transmitting tasks to each user, C4 represents the power limit when transmitting signals to users, C5 and C6 represent the minimum requirements for semantic rate and semantic similarity, respectively, and C7 represents that when transmitting text, semantic relay can only forward information received from the BS to users. n is the average number of semantic symbols transmitted for task n; M is the number of subchannels in the system; P max is the maximum constraint on the transmission power of task n; G th The minimum score requirements for semantic similarity and semantic rate when transferring for task n.

[0077] S3, building and training a hybrid deep reinforcement learning model to obtain a policy network that can achieve optimal resource allocation, and performing wireless resource allocation optimization based on the multi-task resource allocation optimization model;

[0078] It should be noted that, in this embodiment, the system channel signal-to-noise ratio γ, the task data transmission rate And the task transmission semantic similarity ξ is used as the state, the subchannel allocation scheme ρ, the transmission semantic symbol allocation scheme (the average number of transmitted semantic symbols k when transmitting text and image tasks), and the transmission power allocation scheme p are used as actions. Since the transmission power is a continuous variable, the channel allocation vector and the number of transmitted semantic symbols are discrete variables. The reinforcement learning algorithm soft actor critic (SAC) is extended to discrete space and continuous space respectively, and a two-layer hybrid SAC algorithm is constructed to train the intelligent agent and obtain the optimal resource allocation strategy network.

[0079] Specifically, in a feasible implementation manner, the implementation of the above S3 may include the following steps:

[0080] S31, define the state space, action space and reward function of the hybrid deep reinforcement learning model;

[0081] The state space of the system is in, represents the channel signal-to-noise ratio of the system user transmission task and the signal-to-noise ratio from BS to semantic relay, Indicates the transmission rate of the system user transmission task, Indicates the semantic similarity of tasks transferred by system users.

[0082] The action space consists of discrete actions a d = {ρ,k} and continuous action a c ={p}, where ρ = [ρ1,…,ρ M ] T is the subchannel allocation matrix, ρ m=[ρ m,1 ,…,ρ m,N ] is the task allocation channel action, the sub-channel allocation vector dimension is M×N, and p is the task transmission power allocation matrix.

[0083] Based on the goals and constraints, an effective reward function is designed to constrain the behavior of the agent by combining the requirements of semantic transmission efficiency and semantic similarity. Therefore, the reward function is designed as:

[0084]

[0085] Where QoE is the quality of experience of the transmission task; N is the number of end users; and are the penalties for not meeting the requirements of semantic similarity, semantic transmission rate, and semantic relay forwarding rate at the nth user; ν ξ 、 and ν s is a given penalty coefficient; N image is the number of image users; N text is the number of text users. S32, build a two-layer hybrid SAC algorithm model, train the agent, and obtain the optimal resource allocation strategy network;

[0086] Among them, Figure 3 As shown in Figure 1, the two-layer hybrid SAC algorithm model includes a SAC model for discrete action space and a SAC model for continuous action space. The discrete SAC model is used to optimize discrete action variables, including subchannel allocation and the average number of transmitted semantic features, while the continuous SAC model is used to optimize continuous action variables, namely, transmission power. The SAC algorithm introduces maximum entropy to encourage strategy exploration, enhance the agent's search ability, and avoid falling into local optimality. Figure 3 As shown in the figure, each SAC algorithm model contains five neural networks: Actor network, Critic-Q network 1, Critic-Q network 2, Target-Q network 1 and Target-Q network 2.

[0087] The training process of the two-layer hybrid SAC algorithm is as follows:

[0088] Initialize network parameters and clear the experience replay pool;

[0089] Initialize actions according to random strategies and generate random initialization states based on the initial actions;

[0090] The current state Input discrete policy network π θ (·) and continuous action policy network The Actor network of the hybrid SAC algorithm model is based on the current state Get discrete actions and continuous action Agent performs actions Status transferred to Calculate the instant reward r t , will (s t ,a t ,r t ,s t+1 ) is stored in the experience replay pool;

[0091] When the experience replay pool is larger than the storage capacity, Z sequences are sampled from it, the gradient of the network loss function (Q value) is calculated based on the samples, and the network parameters are updated, including: updating the discrete SAC framework policy network parameters, value network parameters and entropy weight parameters; updating the continuous SAC framework policy network parameters, value network parameters and entropy weight parameters.

[0092] Among them, the Actor loss function in the discrete SAC layer is for:

[0093]

[0094] Loss function of the Critic-Q network in the discrete SAC layer for:

[0095]

[0096] in, θ is the parameter of the Actor network in the discrete SAC layer, ψ j is the parameter of the jth Critic-Q network in the discrete SAC layer, ψ j ′ is the parameter of the jth Target-Q network in the discrete SAC layer, γ is the discount factor, and D is the experience replay pool.

[0097] The entropy weight α loss function in the discrete SAC layer is:

[0098]

[0099] Among them, H0 is the target entropy, which is a given hyperparameter.

[0100] Actor loss function in continuous SAC layers for:

[0101]

[0102] Loss function J of the Critic-Q network in continuous SAC layers Q (ψ j )for:

[0103]

[0104] in, are the parameters of the Actor network in the continuous SAC layer, is the parameter of the j-th Critic-Q network in the continuous SAC layer, are the parameters of the j-th Target-Q network in the continuous SAC layer.

[0105] Entropy weight in continuous SAC layer The loss function is:

[0106]

[0107] Then, each iteration, the target policy network and target action network parameters of the hybrid SAC architecture are soft-updated:

[0108] ψ′1=τψ1+(1-τ)ψ′1

[0109] ψ′2=τψ2+(1-τ)ψ′2

[0110]

[0111] When the number of iterations reaches a preset threshold, offline training is completed and a policy network that can achieve optimal resource allocation is obtained. This policy network can be used to realize resource allocation in a multi-task semantic communication system.

[0112] In summary, this embodiment takes into account the advantages of deep reinforcement learning in solving complex resource management problems. In order to improve the performance of multi-task semantic communication networks in device-constrained scenarios, deep reinforcement learning technology is introduced. A multi-task semantic communication system resource allocation method based on deep reinforcement learning is designed. It provides customized services for the multi-task data collaborative transmission network, compresses and transmits task data with high quality and in a personalized manner, which can reduce the semantic network deployment overhead while meeting the user device limitations and achieve efficient utilization of spectrum bandwidth resources.

[0113] Second embodiment

[0114] This embodiment provides an electronic device, such as Figure 4 As shown, the electronic device includes: a processor and a memory; wherein the processor and the memory can be connected via a communication bus; the memory stores at least one instruction, which is loaded and executed by the processor to implement the method of the first embodiment described above. In addition, the electronic device may also include a transceiver; the processor and the transceiver can be connected via a communication bus, and the transceiver is used to communicate with other devices.

[0115] Next, combine Figure 4A detailed introduction to the various components of the electronic device is given below:

[0116] Among them, the processor is the control center of the electronic device, and the electronic device may include multiple processors, each of which may be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). The processor here can be a processor or a general term for multiple processing elements. For example, the processor is one or more central processing units (CPUs), or other general-purpose processors, application specific integrated circuits (ASICs), or one or more integrated circuits configured to implement an embodiment of the present invention, such as one or more microprocessors (digital signal processors, DSPs), or one or more field programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The processor can perform various functions of the electronic device by running or executing software programs stored in the memory and calling data stored in the memory.

[0117] In a specific implementation, as an embodiment, the processor may include one or more CPUs, such as Figure 4 The CPU0 and CPU1 shown in FIG are, of course, only exemplary.

[0118] The memory is used to store the software program for executing the solution of the present invention, and the execution is controlled by the processor. The specific implementation method can refer to the above method embodiment and will not be repeated here.

[0119] Optionally, the memory may be a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (RAM) or other type of dynamic storage device that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compact disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory may be integrated with the processor or exist independently and accessed through the interface circuit ( Figure 4 (not shown) is coupled to the processor, which is not specifically limited in this embodiment of the present invention.

[0120] The transceiver may include a receiver and a transmitter ( Figure 4 The receiver is used to implement the receiving function, and the transmitter is used to implement the sending function. The transceiver can be integrated with the processor or exist independently and communicate with the electronic device through the interface circuit ( Figure 4 (not shown) is coupled to the processor, which is not specifically limited in this embodiment of the present invention.

[0121] In addition, it should be noted that Figure 4 The structure of the electronic device shown in the figure does not constitute a limitation on the device. The actual device may include more or fewer components than shown, or may combine certain components, or arrange the components differently. In addition, the technical effects achieved by the electronic device when executing the method of the first embodiment can refer to the technical effects described in the first embodiment above, and therefore will not be repeated here.

[0122] Third embodiment

[0123] This embodiment provides a computer-readable storage medium storing at least one instruction, which is loaded and executed by a processor to implement the method of the first embodiment described above. The computer-readable storage medium may be a ROM, random access memory, CD-ROM, magnetic tape, floppy disk, or optical data storage device. The instructions stored therein can be loaded by a processor in a terminal to execute the method described above.

[0124] Furthermore, it should be noted that the present invention may be provided as a method, apparatus, or computer program product. Therefore, embodiments of the present invention may take the form of a fully or partially hardware embodiment, a fully or partially software embodiment, or an embodiment combining software and hardware aspects. Furthermore, when implemented using software, embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code. The computer program product comprises one or more computer instructions or computer programs. When the computer instructions or computer program are loaded or executed on a computer, the processes or functions described in accordance with the embodiments of the present invention are fully or partially generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired connection (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium may be any available medium accessible by a computer or a data storage device such as a server or data center that contains a collection of one or more available media. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a DVD), or a semiconductor medium. The semiconductor medium may be a solid state drive.

[0125] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the process in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0126] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device, so that a series of operation steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for implementing the process in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0127] It should also be noted that, in this document, relational terms such as first and second are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any actual relationship or order between these entities or operations. The terms "include," "comprises," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or terminal device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or terminal device. Without further limitation, an element defined by the phrase "comprising a..." does not preclude the presence of other identical elements in the process, method, article, or terminal device comprising the element. In addition, the term "and / or" is merely a description of an associative relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can represent: the presence of A alone, the presence of A and B simultaneously, or the presence of B alone, where A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding. "At least one" means one or more, and "more" means two or more. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can be represented by: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0128] In addition, it can be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0129] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0130] In the several embodiments provided herein, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of functional modules / units is merely a logical functional division. In actual implementation, other division methods may be used, such as multiple units or components being combined or integrated into another device, or some features being ignored or not implemented. Furthermore, the coupling or direct coupling or communication connection shown or discussed between each other may be through some interface, indirect coupling or communication connection between devices or units, and may be electrical, mechanical, or other forms. Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, i.e., they may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of the present embodiment according to actual needs. In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit.

[0131] If the method is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0132] Finally, it should be noted that the above description is merely a preferred embodiment of the present invention. It should be noted that, although preferred embodiments of the present invention have been described, those skilled in the art, once understanding the basic inventive concepts of the present invention, may make various improvements and modifications without departing from the principles of the present invention. Such improvements and modifications should also be considered within the scope of protection of the present invention. Therefore, the appended claims are intended to be interpreted as covering the preferred embodiments and all variations and modifications that fall within the scope of the embodiments of the present invention.

Claims

1. A resource allocation method for a multi-task semantic communication system based on deep reinforcement learning, characterized in that: The multi-task semantic communication system resource allocation method based on deep reinforcement learning includes: Construct a multi-task semantic communication network model assisted by semantic relay; Based on the multi-task semantic communication network model, a multi-task resource allocation optimization model is established with the goal of maximizing the multi-task user experience quality; wherein the multi-task resource allocation optimization model is used to adjust the power allocation, sub-channel allocation, and transmission semantic symbol number allocation of each task to maximize the multi-task user experience quality; Build and train a hybrid deep reinforcement learning model to obtain a policy network that can achieve optimal resource allocation, and optimize wireless resource allocation based on the multi-task resource allocation optimization model; The reward function r of the hybrid deep reinforcement learning model is expressed as: Where QoE is the quality of multi-tasking user experience; N is the number of end users; and are the penalties for not meeting the semantic similarity, semantic transmission rate, and semantic relay forwarding rate requirements at the nth user; v ξ 、 and v s is a given penalty coefficient; N image is the number of image users; N text The number of text users.

2. The resource allocation method for a multi-task semantic communication system based on deep reinforcement learning according to claim 1, characterized in that: The multi-task semantic communication network model includes: a base station, a semantic relay, and terminal users with limited device computing and storage; wherein the terminal users include image users and text users; The base station is equipped with a scene graph generation model for generating interpretable image semantic features and a DeepSC encoder for extracting text semantic information; The DeepSC decoder is deployed on the semantic relay to assist in the decoding and transmission of text information, and transmits the decoding results to the text user through bit encoding; The intelligent tasks in the network include image transmission tasks and text transmission tasks; among them, For image transmission tasks, the base station captures the objects and their relationships in the image through a scene graph generation model, obtains semantic information in an interpretable text format, and further transmits it to the image user through bit encoding; For text transmission tasks, a semantic relay is introduced near the text user to assist in the decoding and forwarding of semantic information. The semantic relay semantically decodes the received text semantic information and transmits it to the text user through bit encoding.

3. The resource allocation method for a multi-task semantic communication system based on deep reinforcement learning according to claim 2, characterized in that: Based on the multi-task semantic communication network model, a multi-task resource allocation optimization model is established with the goal of maximizing the multi-task user experience quality, including: Based on the multi-task semantic communication network model, obtaining the semantic transmission rate and semantic similarity of each task; Based on the semantic transmission rate and semantic similarity of each task, a multi-task resource allocation optimization model is established with the goal of maximizing the multi-task user experience quality.

4. The resource allocation method for a multi-task semantic communication system based on deep reinforcement learning according to claim 3, characterized in that: The semantic similarity of the image transmission task is evaluated by the cosine of the angle between the image vector and its normalized semantic triple vector. The semantic similarity of the text transmission task is obtained by performing data regression and curve fitting on the data of the base station and semantic relay under different transmission semantic symbol numbers and signal-to-noise ratios. The semantic rate is defined as the rate at which users receive task information and is expressed as: in, is the semantic rate of task n; γ n is the signal-to-noise ratio of the transmission channel for task n; B is the channel bandwidth; The signal-to-noise ratio γ of the transmission channel of task n n The calculation formula is: Among them, ρ m,n When task n occupies subchannel m during transmission, p n is the transmission power when task n is transmitted to the user; σ 2 is the noise power; h m,n is the channel gain of the nth task on the mth channel, including large-scale fading and small-scale Rayleigh fading considering path loss and shadowing effects; M is the number of subchannels in the system.

5. The method for allocating resources in a multi-task semantic communication system based on deep reinforcement learning according to claim 4, wherein: The quality of multitasking user experience is expressed as: Among them, QoE represents the quality of multi-task user experience; λ is the user's preference parameter for semantic rate; e is a natural number base; β is the semantic rate score parameter; ε is the semantic similarity score parameter; The minimum semantic rate requirement for task data transmission; ξ req Minimum semantic similarity requirement for task data transmission; is the semantic rate of task n; ξ n is the semantic similarity of task n, is the semantic rate score of task n, is the semantic similarity score of task n.

6. The method for allocating resources in a multi-task semantic communication system based on deep reinforcement learning according to claim 5, wherein: The multi-task resource allocation optimization model P0 is expressed as: in, is the approximate semantic entropy of text transmission; B is the channel bandwidth; N image is the number of image users; N text is the number of text users; k n is the average number of semantic symbols transmitted for task n; M is the number of subchannels in the system; P max is the maximum constraint on the transmission power of task n; G th The minimum score requirements for semantic similarity and semantic rate when transferring for task n.

7. The method for allocating resources in a multi-task semantic communication system based on deep reinforcement learning according to claim 3, wherein: Build and train a hybrid deep reinforcement learning model, including: The reinforcement learning algorithm is extended to discrete space and continuous space respectively, and a two-layer hybrid deep reinforcement learning model is constructed; wherein, the hybrid deep reinforcement learning model takes the system channel signal-to-noise ratio, task transmission semantic rate and task transmission semantic similarity as states, and takes the sub-channel allocation scheme, transmission semantic symbol allocation scheme and transmission power allocation scheme as actions; wherein, the transmission semantic symbol allocation scheme refers to the average number of transmission semantic symbols during text transmission and image transmission; the hybrid deep reinforcement learning model includes a reinforcement learning model for discrete action space and a reinforcement learning model for continuous action space; wherein, the reinforcement learning model for discrete action space is used to optimize discrete action variables, including sub-channel allocation and transmission semantic symbol allocation; the reinforcement learning model for continuous action space is used to optimize continuous action variables, namely transmission power allocation; Train intelligent agents based on a hybrid deep reinforcement learning model to obtain a policy network that achieves optimal resource allocation.

Images