Multi-task semantic communication method and system, and device and storage medium

Abstract

The embodiments of the present application relate to the technical field of communications. Provided are a multi-task semantic communication method and system, and a device and a storage medium. The method comprises: using a feature extraction unit to extract a semantic feature vector of an input signal; performing quantization and modulation to obtain a semantic feature signal; transmitting the semantic feature signal to a base station; the base station generating a transmission signal corresponding to each task receiving end; and the task receiving end performing channel equalization on the transmission signal and then inputting the transmission signal into a semantic inference model for semantic inference, so as to obtain target processing data corresponding to a communication task. At a task sending end, semantic information is extracted into discrete semantic feature vectors in discriminative semantic sub-spaces, and then the base station is used to forward signals; and semantic feature signals corresponding to different communication tasks are encoded in separated sub-spaces, and are approximately orthogonal to each other. Since semantic features of a plurality of users are in discriminative feature domains, interference is significantly reduced, and there is no need to design a complex communication system, thereby reducing the transmission cost and improving the transmission accuracy.

Description

Multi-task semantic communication method, system, device and storage medium TECHNICAL FIELD

[0001] The present application relates to the technical field of communication, in particular to a multi-task semantic communication method, system, device and storage medium. BACKGROUND

[0002] As a new intelligent communication method, semantic communication deeply mines the semantic dimension of data, and realizes high-level abstract representation and intelligent simple transmission of information. In related technologies, semantic communication is mainly single-user semantic communication. If multi-user semantic broadcast communication is to be performed, two terminals that interact with each other are each equipped with a base station, and the two terminals communicate with each other independently. However, a large number of base stations are required in such a communication network, and the channel transmission capacity cannot be fully utilized, resulting in a complex transmission network and high transmission cost. SUMMARY

[0003] The main purpose of the embodiments of the present application is to propose a multi-task semantic communication method, system, device and storage medium, which can reduce the number of base stations and reduce the transmission cost.

[0004] To achieve the above purpose, a first aspect of an embodiment of the present application provides a multi-task semantic communication method applied to a task sending end, the task sending end corresponding to a task receiving end, and used for performing a communication task, the method comprising:

[0005] obtaining a transmission signal corresponding to the communication task, and extracting a semantic feature vector of the transmission signal by using a feature extraction unit;

[0006] quantizing the semantic feature vector to obtain a discrete semantic feature vector, and modulating the discrete semantic feature vector to obtain a semantic feature signal;

[0007] transmitting the semantic feature signal to a base station, so that the base station obtains a transmission signal corresponding to each task receiving end according to the semantic feature signal of different task sending ends, and transmits the transmission signal to a plurality of task receiving ends one by one, the task receiving end performs channel equalization on the transmission signal to obtain an estimated signal, and inputs the estimated signal into a pre-trained semantic reasoning model to perform semantic reasoning, and obtains target processing data corresponding to the communication task.

[0008] In some embodiments, the quantizing the semantic feature vector to obtain a discrete semantic feature vector comprises:

[0009] obtaining a quantization codebook shared between the task sending end and the corresponding task receiving end, the quantization codebook comprising a preset number of quantization vectors;

[0010] divide the semantic feature vector into at least one linear vector, a dimension of the linear vector being consistent with a dimension of the quantization vector;

[0011] obtain the discrete semantic feature vector according to an inner product of the quantization vector and the linear vector.

[0012] In some embodiments, the obtaining the discrete semantic feature vector according to the inner product of the quantization vector and the linear vector comprises:

[0013] perform inner product of the linear vector and the quantization vector respectively to obtain an inner product value, and select an index position of the quantization vector in the quantization codebook corresponding to a minimum inner product value;

[0014] obtain the discrete semantic feature vector based on the index position corresponding to each linear vector.

[0015] In some embodiments, the transmission signal comprises an image to be classified or an image to be reconstructed;

[0016] When the transmission signal is the image to be classified, the semantic inference model is a neural network model, and the target processing data is a classification result of the image to be classified.

[0017] When the transmission signal is the image to be reconstructed, the feature extraction unit is an encoder unit, the semantic inference model is a decoder unit, and the target processing data is a reconstruction result corresponding to the image to be reconstructed.

[0018] In some embodiments, when the feature extraction unit is an encoder unit, the extracting a semantic feature vector of a transmission signal by using the feature extraction unit comprises:

[0019] performing first downsampling and self-attention processing on the transmission signal by using a first encoding block to obtain a first feature map, a width and a height of the first feature map being half of the transmission signal;

[0020] performing second downsampling and self-attention processing on the first feature map by using a second encoding block to obtain a second feature map, a width and a height of the second feature map being half of the first feature map;

[0021] performing third downsampling and self-attention processing on the second feature map by using a third encoding block to obtain the semantic feature vector.

[0022] To achieve the above object, a second aspect of the embodiments of the present application proposes a multi-task semantic communication method applied to a task receiving end, the method comprising:

[0023] obtaining a transmission signal from a base station, the transmission signal being calculated by the base station according to the semantic feature signal obtained by the multi-task semantic communication method of any one of the first aspect;

[0024] performing channel equalization on the transmission signal to obtain an estimated signal;

[0025] inputting the estimated signal into a pre-trained semantic inference model to perform semantic inference, to obtain target processing data corresponding to the communication task.

[0026] In some embodiments, the performing channel equalization on the transmission signal to obtain an estimated signal comprises:

[0027] obtaining channel state information between the task receiving end and the base station, and calculating conjugate transpose data of the channel state information;

[0028] dividing the transmission signal by the conjugate transpose data to obtain the estimated signal.

[0029] In some embodiments, when the semantic inference model is a neural network model, before the inputting the estimated signal into a pre-trained semantic inference model to perform semantic inference, the method further comprises:

[0030] obtaining a training data set, the training data set comprising a transmission sample signal and a data processing label, the transmission sample signal being obtained by the base station according to a semantic sample signal corresponding to each task sending end, the semantic sample signal being obtained according to a transmission sample signal;

[0031] obtaining an estimated sample signal according to the transmission sample signal, and training the semantic inference model using the estimated sample signal to obtain a data prediction result;

[0032] calculating a first loss value according to the data prediction result and the data processing label, a second loss value according to the transmission sample signal and the total semantic sample signal, and a third loss value according to the transmission sample signal and the transmission sample signal;

[0033] obtaining a total loss value according to the first loss value, the second loss value and the third loss value, adjusting the weight of the semantic inference model based on the total loss value to obtain the trained semantic inference model.

[0034] In some embodiments, when the semantic inference model is a decoder unit, before the inputting the estimated signal into a pre-trained semantic inference model to perform semantic inference, the method further comprises:

[0035] obtaining a training data set, the training data set comprising a transmission sample signal and a data processing label, the transmission sample signal being obtained by the base station according to a semantic sample signal corresponding to each task sending end, the semantic sample signal being obtained according to a transmission sample signal;

[0036] obtaining an estimated sample signal according to the transmission sample signal, and training the semantic inference model by using the estimated sample signal to obtain a data prediction result;

[0037] calculating a total loss value according to the data processing label and the data prediction result;

[0038] adjusting the weight of the semantic inference model based on the total loss value to obtain the trained semantic inference model.

[0039] To achieve the above object, a third aspect of the embodiments of the present application proposes a multi-task semantic communication system, comprising:

[0040] a plurality of task sending ends, the task sending end being configured to obtain a semantic feature signal according to the multi-task semantic communication method of any one of the first aspect, and send the semantic feature signal to the base station respectively, each task sending end corresponding to a task receiving end;

[0041] a base station, the base station being configured to obtain the semantic feature vector from a plurality of different task sending ends to obtain a transmission signal corresponding to each task receiving end, and send the transmission signal to a plurality of task receiving ends one by one;

[0042] a plurality of task receiving ends, the task receiving end being configured to obtain a transmission signal, and obtain target processing data corresponding to the communication task according to the transmission signal by using the multi-task semantic communication method of any one of the second aspect.

[0043] In some embodiments, the step of the base station obtaining the semantic feature vector from a plurality of different task sending ends to obtain a transmission signal corresponding to each task receiving end comprises:

[0044] selecting a target sending end from the task sending end one by one, and obtaining a task receiving end corresponding to the target sending end as a target receiving end;

[0045] obtaining target transmission data according to the transmission parameter between the target sending end and the base station and the semantic feature vector of the target sending end;

[0046] obtaining interference transmission data according to the transmission parameter between other task sending ends and the base station and the semantic feature vector of the task sending end, and accumulating the interference transmission data to obtain total interference data;

[0047] According to the target transmission data and the total interference data, the transmission signal of the target receiving end is obtained, and the transmission signal of each task receiving end is obtained one by one.

[0048] To achieve the above object, a fourth aspect of the embodiments of the present application provides an electronic device, the electronic device comprising a memory and a processor, the memory storing a computer program, and the processor implementing the method of the first aspect or the second aspect when executing the computer program.

[0049] To achieve the above object, a fifth aspect of the embodiments of the present application provides a storage medium, the storage medium being a storage medium, the storage medium storing a computer program, and the computer program being executed by a processor to implement the method of the first aspect or the second aspect.

[0050] The multi-task semantic communication method, system, device and storage medium provided by the embodiments of the present application obtain the transmission signal corresponding to the communication task, extract the semantic feature vector of the transmission signal by using the feature extraction unit, quantize the semantic feature vector to obtain the discrete semantic feature vector, modulate the discrete semantic feature vector to obtain the semantic feature signal, and transmit the semantic feature signal to the base station, so that the base station obtains the transmission signal corresponding to each task receiving end according to the semantic feature signal of the different task sending end, transmits the transmission signal to the multiple task receiving ends one by one, the task receiving end performs channel equalization on the transmission signal to obtain the estimated signal, and inputs the estimated signal into the pre-trained semantic reasoning model to perform semantic reasoning, and obtains the target processing data corresponding to the communication task. The embodiments of the present application extract the semantic features of the transmission signal at different task sending ends, extract the semantic information into the discrete semantic feature vector in the distinguishable semantic subspace, then use the same base station to forward the signal, and the semantic feature signals corresponding to different communication tasks are encoded in separate subspaces during the transmission process, and are approximately orthogonal to each other. Then, the task receiving end corresponding to the communication task directly obtains the corresponding estimated signal through channel equalization, and performs semantic reasoning on the estimated signal. Since the semantic features of multiple users are in a distinguishable feature domain, the interference of multiple users can be significantly reduced, a complex communication system does not need to be designed, the transmission cost is reduced, and the transmission accuracy is improved. BRIEF DESCRIPTION OF DRAWINGS

[0051] FIG. 1 is a schematic diagram of a multi-task semantic communication system provided by the embodiments of the present application.

[0052] FIG. 2 is an optional flowchart of a multi-task semantic communication method provided by the embodiments of the present application.

[0053] FIG. 3 is a flowchart of extracting a semantic feature vector of a transmission signal by using a feature extraction unit provided by the embodiments of the present application.

[0054] FIG. 4 is a schematic diagram of a structure of an encoder unit according to an embodiment of the present application.

[0055] FIG. 5 is a flowchart of quantizing a semantic feature vector to obtain a discrete semantic feature vector according to an embodiment of the present application.

[0056] FIG. 6 is a flowchart of obtaining a transmission signal corresponding to each task receiving end from a semantic feature vector from a plurality of different task sending ends according to an embodiment of the present application.

[0057] FIG. 7 is a flowchart of an optional multi-task semantic communication method according to an embodiment of the present application.

[0058] FIG. 8 is a schematic diagram of a training process of a semantic reasoning model according to an embodiment of the present application.

[0059] FIG. 9 is a schematic diagram of a structure of a decoder unit according to an embodiment of the present application.

[0060] FIG. 10 is another schematic diagram of a training process of a semantic reasoning model according to an embodiment of the present application.

[0061] FIG. 11 is a schematic diagram of classification performance of an image to be classified according to an embodiment of the present application.

[0062] FIG. 12 is another schematic diagram of classification performance of an image to be classified according to an embodiment of the present application.

[0063] FIG. 13 is a schematic diagram of reconstruction performance of an image to be reconstructed according to an embodiment of the present application.

[0064] FIG. 14 is another schematic diagram of reconstruction performance of an image to be reconstructed according to an embodiment of the present application.

[0065] FIG. 15 is a schematic diagram of a hardware structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0066] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0067] It should be noted that although the functional modules are divided in the device schematic diagram, and the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a manner different from the module division in the device or the order in the flowchart.

[0068] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in the description herein is for describing particular embodiments only and is not intended to be limiting of the application.

[0069] First, the terms involved in the present application are analyzed:

[0070] Artificial intelligence (AI): is a new technical science of studying, developing theories, methods, technologies and application systems for simulating, extending and expanding human intelligence; artificial intelligence is a branch of computer science, and artificial intelligence tries to understand the essence of intelligence and produce a new intelligent machine that can react in a similar way to human intelligence. The research in this field includes robots, language recognition, image recognition, natural language processing and expert systems. Artificial intelligence can simulate the information process of human consciousness and thinking. Artificial intelligence is also the theory, method, technology and application system of using digital computer or digital computer controlled machine to simulate, extend and expand human intelligence, perceive environment, acquire knowledge and use knowledge to obtain optimal results.

[0071] As a new intelligent communication method, semantic communication deeply mines the semantic dimension of data, realizes the high abstraction of information and the simple transmission of intelligence. In related technologies, semantic communication is mainly single-user semantic communication. If multi-user semantic broadcast communication is to be performed, two terminals interacting with each other are usually equipped with a base station respectively, and the two terminals communicate with each other individually. However, a large number of base stations are required in this communication network, and the channel transmission capacity cannot be fully utilized, resulting in a complex transmission network and high transmission cost.

[0072] Based on this, the embodiments of the present application provide a multi-task semantic communication method, system, device and storage medium. In different task sending ends, the semantic features of the transmitted signals are extracted, and the semantic information is extracted into discrete semantic feature vectors in a distinguishable semantic subspace. Then, the same base station is used to forward the signals. In the sending process, the semantic feature signals corresponding to different communication tasks are encoded in separate subspaces and are approximately orthogonal to each other. Then, the task receiving end corresponding to the communication task directly obtains the corresponding estimated signal through channel equalization and performs semantic reasoning on the estimated signal. Since the semantic features of multiple users are in a distinguishable feature domain, the interference of multiple users can be significantly reduced, a complex communication system does not need to be designed, the transmission cost is reduced, and the transmission accuracy is improved.

[0073] The embodiments of the present application provide a multi-task semantic communication method, system, device and storage medium, which are specifically described as follows. First, the multi-task semantic communication method in the embodiments of the present application is described.

[0074] Embodiments of the present application can acquire and process related data based on artificial intelligence technology. Among them, artificial intelligence (AI) is to use digital computers or digital computer controlled machines to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology of computer science, which tries to understand the essence of intelligence and produce a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines, so that machines have the functions of perception, reasoning and decision-making.

[0075] Artificial intelligence technology is a comprehensive discipline, involving a wide range of fields, both hardware and software level technology. Artificial intelligence basic technology generally includes, such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction system, mechatronics and other technologies. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology and machine learning / deep learning and other fields.

[0076] The multi-task semantic communication method provided by the embodiments of the present application relates to the field of communication technology. The multi-task semantic communication method provided by the embodiments of the present application can be applied in a terminal, can also be applied in a server, and can also be a computer program running in a terminal or a server. For example, the computer program can be a native program or a software module in the operating system; it can be a native application program (APP), that is, a program that needs to be installed in the operating system to run, such as a client supporting multi-task semantic communication, or a small program that only needs to be downloaded into a browser environment to run; it can also be a small program that can be embedded into any APP. In short, the above computer program can be any form of application program, module or plug-in. Among them, the terminal communicates with the server through a network. The multi-task semantic communication method can be executed by the terminal or the server, or cooperatively executed by the terminal and the server.

[0077] In some embodiments, the terminal can be a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart watch, or the like. The server can be a standalone server, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and basic cloud computing services such as big data and artificial intelligence platforms, and the like. The server can also be a service node in a blockchain system, the service nodes in the blockchain system form a peer-to-peer (P2P) network among each other, and the P2P protocol is an application layer protocol running on the transmission control protocol (TCP) protocol. The terminal and the server can be connected through a communication connection mode such as Bluetooth, universal serial bus (USB), or a network, and the present embodiment does not limit this.

[0078] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and the like. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform specific tasks or implement specific abstract data types. The present application can also be practiced in a distributed computing environment, in which tasks are performed by remote processing devices connected by a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0079] It should be noted that in each specific embodiment of the present application, when it is necessary to process relevant data related to the identity or characteristics of the user according to user information, user behavior data, user history data, and user location information, etc., the user's permission or consent will be obtained first, and the collection, use, and processing of these data will comply with relevant laws, regulations, and standards of the country and region. In addition, when the present embodiment needs to obtain sensitive personal information of the user, the user's separate permission or separate consent will be obtained through a pop-up window or a jump to a confirmation page, and after obtaining the separate permission or separate consent of the user, the necessary user-related data for the normal operation of the present embodiment will be obtained.

[0080] First, the multi-task semantic communication system provided in the embodiments of this application is described.

[0081] Referring to Figure 1, which is a schematic diagram of a multi-task semantic communication system provided in an embodiment of this application, the multi-task semantic communication system in Figure 1 is a multi-user semantic multiple access network based on Single-carrier Frequency-Division Multiple Access (SFDMA). It includes multiple task senders, a base station, and multiple task receivers. Each task sender and task receiver corresponds one-to-one, forming a task group. Each task group is used to perform a communication task. It is understood that the communication task can be image reconstruction, image classification, information transmission, etc. The communication tasks of different task groups can be the same or different; this embodiment does not limit this. Figure 1 uses N task groups as an example for illustration. Each task group includes a corresponding task sender and task receiver.

[0082] In one embodiment, each task sender corresponds to a semantic encoder. Taking the i-th task sender as an example, the parameters of the semantic encoder are represented as φ. i It can be based on the transmission signal s from the mission sender. i Obtain semantic feature signal x i Then, the base station receiver simultaneously receives semantic feature signals from multiple task transmitters. These different semantic feature signals are then superimposed to obtain the transmission signal y corresponding to each task receiver. i The base station transmits signal y i Forwarding will transmit the signal y i Send it to the corresponding task receiver.

[0083] Next, the receiving end uses a semantic decoder to decode the transmitted signal, specifically: the received transmitted signal y... i Channel equalization is performed to obtain the estimated signal corresponding to the transmitted signal from the mission transmitter. Then estimate the signal The data is fed into a pre-trained semantic reasoning model for semantic reasoning to obtain the target processing data corresponding to the communication task.

[0084] In one embodiment, the semantic encoder of the task sending end in Figure 1 includes a feature extraction unit, a quantizer, and a digital modulation unit, and the semantic decoder of the task receiving end includes an equalizer and a semantic reasoning model.

[0085] Taking the i-th task sender as an example, its expected task processing data is u. i The task sending end utilizes the feature extraction unit From the transmitted signal si The semantic feature vector a is obtained from i Then the quantizer ψ i (·) For semantic feature vector a i Quantization is performed to obtain the discrete semantic feature vector z. i Then, using the digital modulation unit Θ i (·) for discrete semantic feature vector z i Modulation is performed to obtain semantic feature signal x i Correspondingly, the base station acquires the semantic feature signal x from each task sender. i Then, the transmission signal y corresponding to each task receiver is generated. i Based on the channel state information g between the task receiver and the base station i The transmission signal is sent to the corresponding task receiver. The task receiver uses an equalizer to adjust the transmitted signal y. i Perform channel equalization to obtain the estimated signal. For example, the estimated signal of the receiver of the i-th task is Then, the semantic reasoning model corresponding to the i-th task receiver is used. For the estimated signal Perform semantic reasoning to obtain the target processing data. Theoretically, the target data is processed. And task processing data is u i They are basically the same. Among them, θ i Representation of semantic reasoning model Network parameters.

[0086] The following describes, with reference to Figure 1, a multi-task semantic communication method applied to the task sending end in an embodiment of this application.

[0087] Figure 2 is an optional flowchart of a multi-task semantic communication method provided in an embodiment of this application. The method in Figure 2 may include, but is not limited to, steps 110 to 130. It is also understood that this embodiment does not specifically limit the order of steps 110 to 130 in Figure 2, and the order of steps can be adjusted or some steps can be reduced or added according to actual needs.

[0088] Step 110: Obtain the transmission signal corresponding to the communication task, and use the feature extraction unit to extract the semantic feature vector of the transmission signal.

[0089] In one embodiment, due to different communication tasks, the transmitted signal includes either an image to be classified or an image to be reconstructed. The image to be classified is used for image classification, and the image to be reconstructed is used for image reconstruction. When the transmitted signal is an image to be classified, the semantic reasoning model at the task receiver is a neural network model, and the target processing data is the classification result of the image to be classified. When the transmitted signal is an image to be reconstructed, the feature extraction unit at the task sender is an encoder unit, the semantic reasoning model at the task receiver is a decoder unit, and the target processing data is the reconstruction result corresponding to the image to be reconstructed.

[0090] In one embodiment, when the transmitted signal is an image to be classified, for the i-th task sender, the feature extraction unit... It can be a neural network model, capable of analyzing the emitted signal s i Features are extracted and encoded to obtain semantic feature vector a. i This feature extraction process can be represented as:

[0091] In one embodiment, when the transmitted signal is an image to be reconstructed, the feature extraction unit is an encoder unit. Referring to Figure 3, Figure 3 is a flowchart of extracting semantic feature vectors of the transmitted signal using a feature extraction unit according to an embodiment of this application, specifically including:

[0092] Step 310: After performing the first downsampling and self-attention processing on the transmitted signal using the first coding block, the first feature map is obtained.

[0093] In one embodiment, when the transmitted signal is the image to be reconstructed, the transmitted signal is represented as follows: Where H and W represent the height and width of the transmitted signal, and C represents the number of channels for the transmitted signal.

[0094] Referring to Figure 4, which is a schematic diagram of the encoder unit in this embodiment, the encoder unit in Figure 4 includes three encoding blocks: a first encoding block, a second encoding block, and a third encoding block. All three blocks are Swin Transformer structures, each including an encoding patch block and a Swin Transformer encoding block. The encoding patch block divides the input image into multiple non-overlapping patches, which are then fed into the Swin Transformer encoding block. For each patch, self-attention is calculated, and information is integrated through shift operations, thus achieving the encoding process of the input image. The Swin Transformer structure utilizes the encoding patch block and the Swin Transformer encoding block in collaboration to extract and transform semantic information from the image.

[0095] In an embodiment, the first encoding block first performs a first down-sampling on the transmission signal s i to divide the transmission signal into non-overlapping patches (Patch) which are labeled in the order from top-left to bottom-right to obtain a serial sequence and embed it into Embedding, and then perform self-attention processing through the Swin Transformer block to obtain a first feature map, denoted as: The width and height of the first feature map are both half of the transmission signal, and the channel number C1 is consistent with the channel number of the transmission signal.

[0096] Step 320: After the first feature map is subjected to a second down-sampling and self-attention processing by the second encoding block, a second feature map is obtained.

[0097] In an embodiment, then in the second encoding block, the first feature map is first subjected to a second down-sampling by the encoding patch block to reduce the size of the first feature map, reduce the number of patches, thereby reducing the image resolution and adjusting the channel number, which helps to reduce the subsequent calculation amount. After the patch processing, the height and width of the feature map are both reduced to half of the original, while the channel number is expanded to twice the original. Subsequently, the adjusted patch is processed again by the Swin Transformer encoding block to output a second feature map, denoted as: The width and height of the second feature map are both half of the first feature map, and the channel number C2 is twice the channel number C1 of the first feature map.

[0098] Step 330: After the second feature map is subjected to a third down-sampling and self-attention processing by the third encoding block, a semantic feature vector is obtained.

[0099] In an embodiment, the second feature map continues to be subjected to a third down-sampling by the encoding patch block of the third encoding block, and then is processed by the Swin Transformer encoding block to obtain a final feature map, whose width and height are both half of the second feature map, and the channel number C3 is twice the channel number C2 of the second feature map. Finally, the feature map is encoded to output a semantic feature vector The semantic feature vector contains key semantic information in the transmission signal, and the size is significantly reduced compared to the transmission signal.

[0100] Overall, the feature extraction unit that is, the encoder unit, extracts the semantic feature vector of the transmission signal, and the process is represented as:

[0101] a i = Layer3(Layer2(Layer1(s i )))

[0102] wherein Layer1 represents the first encoding block, Layer2 represents the second encoding block, and Layer3 represents the third encoding block.

[0103] Step 120: quantizing the semantic feature vector to obtain a discrete semantic feature vector, and modulating the discrete semantic feature vector to obtain a semantic feature signal.

[0104] In an embodiment, in order to improve the transmission efficiency of the communication system and simplify the processing procedure of the base station, the embodiment of the present application quantizes the semantic feature vector a i , and quantizes the semantic feature vector by using a quantizer to obtain a discrete semantic feature vector. Referring to FIG. 5, which is a flowchart of quantizing the semantic feature vector to obtain a discrete semantic feature vector according to an embodiment of the present application, the embodiment specifically includes the following steps:

[0105] Step 510: obtaining a quantization codebook shared between the task sending end and the corresponding task receiving end.

[0106] The quantization codebook is a learnable codebook, that is, it can be adjusted according to the training process, and it includes a preset number of quantization vectors. The quantization codebook M i shared in the ith task group is represented as:

[0107] wherein K represents the preset number, d represents the dimension of each quantization vector, and m i,j represents the jth quantization vector in the ith quantization codebook.

[0108] Step 520: dividing the semantic feature vector into at least one linear vector.

[0109] The dimension of the linear vector is consistent with the dimension d of the quantization vector, and thus the linear vector a i obtained by dividing the ith semantic feature vector a i is represented as:

[0110] Step 530: obtaining a discrete semantic feature vector according to the inner product of the quantization vector and the linear vector.

[0111] In an embodiment, first, the linear vector is respectively carried out with the quantization vector to obtain the inner product value, the index position of the quantization vector corresponding to the minimum inner product value is selected in the quantization codebook, and the discrete semantic feature vector is obtained based on the index position corresponding to each linear vector. That is, each linear vector is respectively carried out with the vector inner product of the quantization vector, and then the corresponding inner product value is obtained. It can be understood that each linear vector corresponds to K inner product values, and the quantization vector corresponding to the minimum inner product value is selected as the index vector, and the position of the index vector in the quantization codebook is taken as the index position z i,j . In this way, the index position of each linear vector is obtained, and then the discrete semantic feature vector is obtained according to the index position. Assuming that there are n linear vectors, the discrete semantic feature vector z i is expressed as:

[0112] Wherein, Ψ i (·) represents the quantizer of the i-th task sending end.

[0113] In an embodiment, after the discrete semantic feature vector z i is obtained, the digital modulator is further used to modulate the discrete semantic feature vector z i to obtain the semantic feature signal x i , so as to convert the bit stream into a signal form suitable for transmission on the channel. For example, 4PSK modulation can be used, which is expressed as: x i = Θ i (z i ), i∈{1,..,N}

[0114] Step 130: transmitting the semantic feature signal to the base station.

[0115] In an embodiment, after the base station receives the semantic feature signals x i transmitted by different task sending ends, the transmission signal corresponding to each task receiving end is obtained according to the semantic feature signals of different task sending ends. The transmission signal is sent to the multiple task receiving ends one by one, and then the task receiving end carries out channel equalization on the transmission signal to obtain an estimated signal, and inputs the estimated signal into a pre-trained semantic inference model to carry out semantic inference, so as to obtain the target processing data corresponding to the communication task.

[0116] In an embodiment, referring to FIG. 6, FIG. 6 is a flowchart of the base station obtaining the semantic feature vector from multiple different task sending ends to obtain the transmission signal corresponding to each task receiving end according to the embodiments of the present application, which specifically includes:

[0117] Step 610: selecting a target sending end from the task sending end one by one, and taking the task receiving end corresponding to the target sending end as a target receiving end.

[0118] For example, the ith task sender is selected as the target sender, and the task receiver in the ith task group is selected as the target receiver.

[0119] Step 620: obtaining the target transmission data according to the transmission parameter between the target sender and the base station and the semantic feature vector of the target sender.

[0120] In an embodiment, the transmission parameter is calculated according to the channel fading coefficient and the transmission power between the target sender and the base station. For the ith task sender, g i represents the channel fading coefficient between the target sender and the base station, that is, the channel state information, p i is the transmission power of the target sender. At this time, the transmission parameter is represented as:

[0121] Therefore, the target transmission data is represented as:

[0122] Step 630: obtaining the interference transmission data according to the transmission parameter between the other task senders and the base station and the semantic feature vector of the task sender, and accumulating the interference transmission data to obtain the total interference data.

[0123] In an embodiment, the interference transmission data of the jth task sender relative to the ith task sender is represented as: Therefore, the total interference data is represented as:

[0124] Step 640: obtaining the transmission signal of the target receiver according to the target transmission data and the total interference data, and obtaining the transmission signal of each task receiver one by one.

[0125] wherein the transmission signal y i of the ith task receiver is represented as:

[0126] wherein n i is a complex noise vector with mean 0 and variance δ 2 , satisfying

[0127] The embodiment of the present application extracts semantic features of the transmission signal at different task sending ends, extracts semantic information into discrete semantic feature vectors in a distinguishable semantic subspace, and then uses the same base station to forward the signal. In the sending process, the semantic feature signals corresponding to different communication tasks are encoded in separate subspaces and are approximately orthogonal to each other. Then, the task receiving end corresponding to the communication task directly obtains the corresponding estimated signal through channel equalization and performs semantic reasoning on the estimated signal. Since the semantic features of multiple users are in a distinguishable feature domain, the interference of multiple users can be significantly reduced, a complex communication system does not need to be designed, the transmission cost is reduced, and the transmission accuracy is improved.

[0128] The multi-task semantic communication method applied to the task receiving end in the embodiment of the present application will be described below with reference to FIG. 1.

[0129] FIG. 7 is an optional flowchart of the multi-task semantic communication method provided by the embodiment of the present application. The method in FIG. 7 can include but is not limited to steps 710 to 740. It can be understood that the order of steps 710 to 730 in FIG. 7 is not limited in the embodiment, and the order of steps can be adjusted or some steps can be reduced or added according to actual needs.

[0130] Step 710: Obtain a transmission signal from a base station.

[0131] In an embodiment, the transmission signal is calculated from the semantic feature signal obtained by the multi-task semantic communication method of any one of the above embodiments.

[0132] Step 720: Perform channel equalization on the transmission signal to obtain an estimated signal.

[0133] In an embodiment, the channel equalization is performed in an ideal state, that is, it is assumed that the channel state information between each task receiving end and the base station is known information. It can be understood that if the channel state information is unknown, channel estimation needs to be performed at this time, and common channel estimation methods such as LS channel estimation, MMSE, LMMSE, etc. can be used, and the embodiment is not limited in this regard.

[0134] The process of performing channel equalization on the transmission signal to obtain the estimated signal specifically includes: obtaining the channel state information between the task receiving end and the base station, calculating the conjugate transpose data of the channel state information, and dividing the transmission signal by the conjugate transpose data to obtain the estimated signal.

[0135] The estimated signal of the i th task receiving end is denoted as

[0136] In the above formula, since the channel state information g i ​It is known that in the channel equalization process, embodiments of the present application can divide the transmission signal by This can convert the channel effect from multiplication operation to additive operation, thereby reducing the learning burden of the subsequent semantic inference model.

[0137] Step 730: input the estimated signal into the pre-trained semantic inference model for semantic inference to obtain target processing data corresponding to the communication task.

[0138] In an embodiment, when the transmitted signal is an image to be classified, the semantic inference model is a neural network model, and the target processing data is the classification result of the image to be classified. When the transmitted signal is an image to be reconstructed, the feature extraction unit is an encoder unit, the semantic inference model is a decoder unit, and the target processing data is a reconstruction result corresponding to the image to be reconstructed.

[0139] In an embodiment, after the signal equalization processing, the estimated signal is sent to the semantic inference network The semantic inference network obtains semantic features therefrom, performs classification processing or image reconstruction processing thereon, and outputs the final result to obtain target processing data The inference process is represented as:

[0140] wherein θ i represents the parameters of the semantic inference network of the i-th task receiving end.

[0141] The semantic inference model for different types of transmitted signals and the corresponding training process are described below.

[0142] In an embodiment, when the transmitted signal is an image to be classified, the semantic inference model is a neural network model. Referring to FIG. 8, FIG. 8 is a schematic diagram of the training process of the semantic inference model according to an embodiment of the present application, which specifically includes:

[0143] Step 810: obtain a training data set.

[0144] First, the transmitted sample signal generated by each task sending end and the corresponding data processing label are obtained, and the corresponding semantic sample signal is obtained through feature extraction, quantization and modulation, and then the base station obtains the transmission sample signal of each task receiving end according to the semantic sample signal of different task sending ends. The data processing label is the processing data expected to be obtained by the transmitted sample signal.

[0145] In an embodiment, there are L samples in the training data set, corresponding to L transmission sample signals, and the semantic encoder of each task sending end samples the semantic feature signal M times, that is, for each semantic sample signal corresponding to a transmission sample signal, the semantic encoder divides the semantic sample signal into M discrete data points, and correspondingly, the transmission sample signal also corresponds to M discrete transmission data, and the data processing label also corresponds to M discrete label data. The sampling number here will affect the complexity and accuracy of data processing of the task sending end and the task receiving end. In this embodiment, the sampling number is set according to actual needs.

[0146] Step 820: obtaining an estimated sample signal according to the transmission sample signal, and training the semantic inference model by using the estimated sample signal to obtain a data prediction result.

[0147] In an embodiment, the estimated sample signal is obtained according to the transmission sample signal in the manner of calculating the estimated signal as described above, and the estimated sample signal is input into the semantic inference model for prediction to obtain a data prediction result.

[0148] Step 830: obtaining a first loss value according to the data prediction result and the data prediction result, obtaining a second loss value according to the transmission sample signal and the semantic sample signal, and obtaining a third loss value according to the transmission sample signal and the transmission sample signal.

[0149] In the embodiment of the present application, the training process of the semantic inference model can be regarded as an optimization problem, and the objective function is represented as:

[0150] Wherein, ω i >0 is a weight factor that can be adjusted, I(U i ;Y i ') represents the first loss value, which is the correlation between the data prediction result Y i ' and the data processing label Ui, I(X;Y i ) represents the second loss value, which is the uncertainty between the transmission sample signal Yi and the total semantic sample signal X of the plurality of task sending ends, I(S i ;Y i ) represents the third loss value, which is the correlation between the transmission sample signal Yi and the transmission sample signal Si, and I(X;Y i )-I(S i ;Y i ) can ensure that the balance between the downstream task performance and the communication overhead is met in the communication process.

[0151] In order to develop a semantic encoder with high robustness, it is assumed that the i th transmission sample signal is s i ', and the data processing label is u iX i x′, Y i y i The above formula can be equivalent to:

[0152] where φ i represents the weight of the ith feature extraction unit, represents the expectation of the probability distribution under the condition of s i ′ and u i ′, represents the expectation of the probability distribution under the condition of s i ′ and y i ′, represents the expectation under the condition of x′ and y i ′, represents the probability distribution between x′ and y i ′, H(Y i ′|x′) represents the information entropy between x′ and Y i ′. represents the information entropy between s i ′ and Y i ′.

[0153] Since the calculation of the posterior distribution function involves high-dimensional integration, the embodiments of the present application introduce two approximate distributions and to approximate the high-dimensional distributions and θ KL are parameters that can be learned by the semantic reasoning model. Therefore, the first term of the above formula can be simplified as:

[0154] where D i ≥ 0 represents the DL divergence.

[0155] Next, since the transmission sample signal Y i ′ is a transmission representation destroyed by added Gaussian noise, the lower bound of the entropy function can be derived, that is:

[0156] Therefore, the objective function in the embodiments of the present application can be converted to:

[0157] where and the conditional entropy term in H(Y i ′|x′) are analytically calculated with respect to the parameter φ

[0158] When the semantic encoder is a DNN-based encoder, the channel model can be derived according to the definition of the factorial distribution, where the factorial distribution is defined as: Therefore, the channel model is derived from the conditional distribution p(s i ′,u i ′).

[0159] And the entropy function can be decomposed as follows:

[0160] Therefore, if M batches of data For each small batch of data Pass through the channel noise L-point sampling, the objective function in the embodiment of the present application can be written as:

[0161] Where x j (m,l) represents the semantic sample signal in the mth small batch of data, y i (m,l) represents the transmission sample signal in the mth small batch of data, both of which are discrete feature vectors.

[0162] Step 840: obtaining a total loss value according to the first loss value, the second loss value and the third loss value, adjusting the weight of the semantic inference model based on the total loss value to obtain a trained semantic inference model corresponding to each terminal.

[0163] In an embodiment, the total loss value, i.e., the above-mentioned objective function, is represented as:

[0164] The design of the total loss value in the embodiment of the present application achieves a trade-off between the amount of transmitted semantic information, inter-user interference and inference accuracy, so that the weight of the semantic inference model is adjusted according to the total loss value until the preset training termination condition is reached, i.e., a trained semantic inference model suitable for different communication tasks can be obtained.

[0165] In an embodiment, when the transmitted signal is an image to be reconstructed, the semantic inference model is a decoder unit. Referring to FIG. 9, FIG. 9 is a structural schematic diagram of the decoder unit provided in the embodiments of the present application. In combination with FIG. 4, it can be known from FIG. 9 that the decoder unit has a symmetrical structure with the encoder unit, and includes three decoding blocks, i.e., a first decoding block, a second decoding block and a third decoding block. The first decoding block, the second decoding block and the third decoding block are all Swin Transformer structures, and each includes a Swin Transformer decoding block and a decoding patch block. The Swin Transformer decoding block corresponds to the Swin Transformer encoding block, is used for decoding, and reconstructs the signal. The decoding patch block is used for up-sampling the input image. Therefore, in the embodiments of the present application, the estimated signal Firstly, the first decoding block is entered, and after reconstruction and up-sampling, the signal is changed from to Then, the second decoding block is entered, and after reconstruction and up-sampling, the signal is changed from to Finally, the third decoding block is entered, and after reconstruction and up-sampling, the signal is changed from to HxWx3. At this time, the target processing data is the reconstruction result corresponding to the image to be reconstructed The whole process is represented as:

[0166] Among them, Layer1' represents the first decoding block, Layer2' represents the second decoding block, and Layer3' represents the third decoding block.

[0167] The training process of the decoder unit is described below. In an embodiment, referring to FIG. 10, FIG. 10 is another schematic diagram of the training process of the semantic inference model provided in the embodiments of the present application, and specifically includes the following steps:

[0168] Step 1010: Obtain a training data set.

[0169] Among them, the training data set is similar to the generation mode of the training data set in step 810.

[0170] Step 1020: Obtain an estimated sample signal according to the transmitted sample signal, and train the semantic inference model by using the estimated sample signal to obtain a data prediction result.

[0171] Step 1030: Calculate a total loss value according to the data processing label and the data prediction result.

[0172] Among them, the total loss value is represented as:

[0173] Among them, denotes a total loss value corresponding to the to-be-reconstructed image, n denotes a sample number, s i denotes a data processing label in image reconstruction, denotes a data prediction result in image reconstruction.

[0174] Step 1040: based on the total loss value, weight adjustment is performed on the semantic inference model, to obtain a trained semantic inference model corresponding to each terminal.

[0175] In an embodiment, based on the total loss value, weight adjustment is performed on the semantic inference model until a preset training termination condition is reached, to obtain the trained semantic inference model. The training termination condition here can be that the trained semantic inference model satisfies an evaluation index MS-SSIM. MS-SSIM is a multi-scale structural similarity, which is an index for measuring image reconstruction quality, and the similarity between images is evaluated by considering the structural similarity at different scales.

[0176] In an embodiment, referring to FIG. 11, FIG. 11 is a schematic diagram of classification performance of to-be-classified images provided by an embodiment of the present application. In FIG. 11, image classification is performed for two users U1 and U2, where the quantization bit number is d = 16 and the quantization bit number d = 64, and the training signal-to-noise ratio is 5 dB. In the figure, the vertical axis represents the classification accuracy (percentage), the horizontal axis represents the signal-to-noise ratio (dB), Upper bound represents the performance upper bound, DeepJSCC represents a deep joint source channel coding scheme in the related art, and SFDMA represents the multi-task semantic communication method provided by an embodiment of the present application. As can be seen from FIG. 11, the classification accuracy of the SFDMA scheme provided by an embodiment of the present application is always better than that of the Deep JSCC scheme in the related art, and tends to approach the performance upper bound Upper bound corresponding to different users, proving the effectiveness of the multi-task semantic communication method in image classification.

[0177] In an embodiment, referring to FIG. 12, FIG. 12 is another schematic diagram of classification performance of to-be-classified images provided by an embodiment of the present application. In the figure, handwritten digits of different users need to be recognized. According to the recognition result, the semantic feature signals of users U1, U2 and U3 are separated on the semantic feature subspace, and the classification accuracy of each user is above 94%.

[0178] In an embodiment, referring to FIG. 13, FIG. 13 is a schematic diagram of reconstruction performance of to-be-reconstructed images provided by an embodiment of the present application. FIG. 13 is a visualization of the semantic feature subspace. As can be seen from FIG. 13, the images of two users U1 and U2 can be separated on the semantic feature subspace, and the reconstruction process will not be disturbed by other users.

[0179] In an embodiment, referring to FIG. 14, FIG. 14 is another schematic diagram of reconstruction performance of a to-be-reconstructed image provided in the embodiment of the present application. The reconstruction performance comparison when the peak signal-to-noise ratio (PSNR) is used as an evaluation index is shown in FIG. 14. As can be seen from FIG. 14, the reconstruction performance of the SFDMA scheme provided in the embodiment of the present application is always superior to that of the DeepJSCC scheme in the related art, and tends to the performance upper bound, proving the effectiveness of the multi-task semantic communication method in the embodiment of the present application in image reconstruction.

[0180] The embodiment of the present application improves the communication performance in a multi-user broadcast communication scenario. The semantic feature signals of different users are approximately orthogonal, and the semantic decoder of the corresponding task receiver only decodes the relevant information of the task transmitter corresponding thereto to obtain the target processing data, and does not decode the remaining semantic feature signals of the broadcast channel, thereby improving the communication performance of the multi-user broadcast communication in the task-oriented communication. Moreover, the decoding of the corresponding semantic feature signal by each task receiver can enhance the confidentiality of the multi-user semantic broadcast communication transmission and improve the transmission security performance.

[0181] The technical scheme provided in the embodiment of the present application acquires a transmission signal corresponding to a communication task, extracts a semantic feature vector of the transmission signal by using a feature extraction unit, quantizes the semantic feature vector to obtain a discrete semantic feature vector, modulates the discrete semantic feature vector to obtain a semantic feature signal, transmits the semantic feature signal to a base station, so that the base station obtains a transmission signal corresponding to each task receiver according to the semantic feature signals of different task transmitters, and sequentially sends the transmission signals to the plurality of task receivers. The task receiver performs channel equalization on the transmission signal to obtain an estimated signal, and inputs the estimated signal into a pre-trained semantic reasoning model to perform semantic reasoning, thereby obtaining target processing data corresponding to the communication task. In the embodiment of the present application, the semantic features of the transmission signals are extracted by different task transmitters, the semantic information is extracted into the discrete semantic feature vectors in the distinguishable semantic subspace, and then the same base station is used to forward the signals. The semantic feature signals corresponding to different communication tasks are encoded in separate subspaces during the transmission process, and are approximately orthogonal to each other. Then, the task receiver corresponding to the communication task directly acquires the corresponding estimated signal through channel equalization, and performs semantic reasoning on the estimated signal. Since the semantic features of the multi-users are in the distinguishable feature domain, the interference of the multi-users can be significantly reduced, a complex communication system does not need to be designed, the transmission cost is reduced, and the transmission accuracy is improved.

[0182] The embodiment of the present application also provides an electronic device, which comprises:

[0183] at least one memory;

[0184] at least one processor;

[0185] at least one program;

[0186] The program is stored in the memory, and the processor executes the at least one program to implement the multi-task semantic communication method described above. The electronic device can be any intelligent terminal including a mobile phone, a tablet computer, a personal digital assistant (PDA), a vehicle-mounted computer, etc.

[0187] Referring to FIG. 15, FIG. 15 shows the hardware structure of an electronic device according to another embodiment, which includes:

[0188] The processor 1501 can be implemented in a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, etc., and is configured to execute related programs to implement the technical solutions provided by the embodiments of the present application.

[0189] The memory 1502 can be implemented in a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM), etc. The memory 1502 can store an operating system and other application programs. When the technical solutions provided by the embodiments of the present application are implemented by software or firmware, the related program codes are stored in the memory 1502 and are called and executed by the processor 1501 to implement the multi-task semantic communication method of the embodiments of the present application.

[0190] The input / output interface 1503 is configured to implement information input and output.

[0191] The communication interface 1504 is configured to implement the communication interaction between the device and other devices. The communication can be realized by wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0192] The bus 1505 is configured to transmit information between various components (such as the processor 1501, the memory 1502, the input / output interface 1503, and the communication interface 1504) of the device.

[0193] The processor 1501, the memory 1502, the input / output interface 1503, and the communication interface 1504 are connected to each other through the bus 1505 to realize the communication connection between them inside the device.

[0194] The embodiment of the present application also provides a storage medium, the storage medium is a storage medium, the storage medium stores a computer program, and the computer program is executed by a processor to realize the multi-task semantic communication method.

[0195] The memory, as a non-transitory storage medium, can be used to store non-transitory software programs and non-transitory computer executable programs. In addition, the memory can include a high-speed random access memory and can also include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state memory device. In some embodiments, the memory can optionally include a memory remotely arranged relative to the processor, and these remote memories can be connected to the processor through a network. Examples of the above network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0196] The multi-task semantic communication method, system, device and storage medium provided by the embodiment of the present application obtain a transmission signal corresponding to a communication task, extract a semantic feature vector of the transmission signal by using a feature extraction unit, quantize the semantic feature vector to obtain a discrete semantic feature vector, modulate the discrete semantic feature vector to obtain a semantic feature signal, and transmit the semantic feature signal to a base station, so that the base station obtains a transmission signal corresponding to each task receiving end according to the semantic feature signals of different task sending ends, transmits the transmission signals to the plurality of task receiving ends one by one, the task receiving end performs channel equalization on the transmission signal to obtain an estimated signal, and inputs the estimated signal into a pre-trained semantic reasoning model to perform semantic reasoning, and obtains target processing data corresponding to the communication task. The embodiment of the present application extracts the semantic feature of the transmission signal at different task sending ends, extracts the semantic information into the discrete semantic feature vector in the distinguishable semantic subspace, and then uses the same base station to forward the signal. The semantic feature signals corresponding to different communication tasks are encoded in separate subspaces during the transmission process, and are approximately orthogonal to each other. Then, the task receiving end corresponding to the communication task directly obtains the corresponding estimated signal through channel equalization, and performs semantic reasoning on the estimated signal. Since the semantic features of multiple users are in a distinguishable feature domain, the interference of multiple users can be significantly reduced, a complex communication system does not need to be designed, the transmission cost is reduced, and the transmission accuracy is improved.

[0197] The embodiments described in the embodiments of the present application are used to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. Those skilled in the art can know that, with the evolution of technology and the appearance of new application scenarios, the technical solutions provided by the embodiments of the present application are also applicable to similar technical problems.

[0198] Those skilled in the art can understand that the technical solutions shown in the figures do not constitute a limitation to the embodiments of the present application, and can include more or fewer steps than the figures, or combine certain steps, or different steps.

[0199] The device embodiments described above are merely illustrative, wherein the units described as separate components can or can not be physically separated, that is, can be located in one place, or can be distributed to multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiments.

[0200] Those skilled in the art can understand that all or some steps in the above disclosed method, functions of the modules / units in the system and the device can be implemented as software, firmware, hardware and appropriate combinations thereof.

[0201] The terms "first", "second", "third", "fourth" and the like (if any) in the specification of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not necessarily limit to those clearly listed steps or units, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0202] It should be understood that in the present application, "at least one" means one or more, and "multiple" means two or more. "And / or" is used to describe the association between the associated objects, which means that there can be three relationships, for example, "A and / or B" can represent three cases: only A, only B, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects. "At least one of the following" or similar expressions means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0203] In several embodiments provided in the present application, it should be understood that the disclosed apparatus and method can be implemented by other manners. For example, the apparatus embodiments described above are merely illustrative, for example, the division of the above units is merely a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units or components shown or discussed can be indirect coupling or communication connection through some interfaces, apparatuses or units, and can be electrical, mechanical or other forms.

[0204] The units described above as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they can be located in one place or distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0205] In addition, the functional units in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0206] If the integrated unit is realized in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the part of the prior art that contributes to the technical solutions or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method of each embodiment of the present application. The foregoing storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk, and various program storage media.

[0207] The preferred embodiments of the embodiments of the present application are described above with reference to the accompanying drawings, but this does not limit the scope of the embodiments of the present application. Any modifications, equivalent replacements and improvements made by those skilled in the art without departing from the scope and essence of the embodiments of the present application shall be within the scope of the embodiments of the present application.

Claims

1. A multi-task semantic communication method, characterized in that, Applied to a task sending end, wherein the task sending end corresponds to a task receiving end, and is used to execute communication tasks, the method includes: The transmission signal corresponding to the communication task is obtained, and the semantic feature vector of the transmission signal is extracted using the feature extraction unit; The semantic feature vector is quantized to obtain a discrete semantic feature vector, and the discrete semantic feature vector is modulated to obtain a semantic feature signal; The semantic feature signal is transmitted to the base station, so that the base station obtains the transmission signal corresponding to each task receiving end based on the semantic feature signal of different task sending ends, and sends the transmission signal to multiple task receiving ends one by one. The task receiving end performs channel equalization on the transmission signal to obtain an estimated signal, and inputs the estimated signal into a pre-trained semantic reasoning model for semantic reasoning to obtain target processing data corresponding to the communication task.

2. The multi-task semantic communication method according to claim 1, characterized in that, The process of quantizing the semantic feature vector to obtain a discrete semantic feature vector includes: Obtain the quantization codebook shared between the task sending end and the corresponding task receiving end, the quantization codebook including a preset number of quantization vectors; The semantic feature vector is divided into at least one linear vector, the dimension of which is the same as the dimension of the quantization vector; The discrete semantic feature vector is obtained by the inner product of the quantization vector and the linear vector.

3. The multi-task semantic communication method according to claim 2, characterized in that, The step of obtaining the discrete semantic feature vector based on the inner product of the quantized vector and the linear vector includes: The inner product value is obtained by taking the inner product of the linear vector with the quantization vector respectively, and the index position of the quantization vector corresponding to the smallest inner product value is selected in the quantization codebook. The discrete semantic feature vector is obtained based on the index position corresponding to each of the linear vectors.

4. The multi-task semantic communication method according to claim 1, characterized in that, The transmitted signal includes an image to be classified or an image to be reconstructed. When the transmitted signal is an image to be classified, the semantic reasoning model is a neural network model, and the target processing data is the classification result of the image to be classified; When the transmitted signal is an image to be reconstructed, the feature extraction unit is an encoder unit, the semantic reasoning model is a decoder unit, and the target processing data is the reconstruction result corresponding to the image to be reconstructed.

5. The multi-task semantic communication method according to claim 4, characterized in that, When the feature extraction unit is an encoder unit, the step of extracting the semantic feature vector of the transmitted signal using the feature extraction unit includes: After performing a first downsampling and self-attention processing on the transmitted signal using the first coding block, a first feature map is obtained, wherein the width and height of the first feature map are both half of the transmitted signal. After performing a second downsampling and self-attention processing on the first feature map using the second coding block, a second feature map is obtained, the width and height of which are both half of the first feature map; The semantic feature vector is obtained by performing a third downsampling and self-attention processing on the second feature map using a third coding block.

6. A multi-task semantic communication method, characterized in that, Applied to a task receiving end, the method includes: Acquire a transmission signal from a base station, the transmission signal being calculated by the base station from the semantic feature signal obtained by the multi-task semantic communication method according to any one of claims 1 to 5; Channel equalization is performed on the transmitted signal to obtain the estimated signal; The estimated signal is input into a pre-trained semantic reasoning model for semantic reasoning to obtain target processing data corresponding to the communication task.

7. The multi-task semantic communication method according to claim 6, characterized in that, The process of performing channel equalization on the transmitted signal to obtain the estimated signal includes: Obtain the channel state information between the task receiving end and the base station, and calculate the conjugate transpose of the channel state information; The estimated signal is obtained by dividing the transmitted signal by the conjugate transpose data.

8. The multi-task semantic communication method according to claim 6, characterized in that, When the semantic reasoning model is a neural network model, before inputting the estimated signal into the pre-trained semantic reasoning model for semantic reasoning, the method further includes: A training dataset is obtained, which includes transmission sample signals and data processing labels. The transmission sample signals are obtained by the base station based on the semantic sample signals corresponding to each task sender. The semantic sample signals are obtained based on the transmission sample signals. Based on the transmitted sample signal, an estimated sample signal is obtained, and the estimated sample signal is used to train the semantic reasoning model to obtain the data prediction result; A first loss value is calculated based on the data prediction results and the data processing labels; a second loss value is calculated based on the transmitted sample signals and the total semantic sample signals; and a third loss value is calculated based on the transmitted sample signals and the transmitted sample signals. The total loss value is obtained based on the first loss value, the second loss value, and the third loss value. The semantic reasoning model is then weighted based on the total loss value to obtain the trained semantic reasoning model.

9. The multi-task semantic communication method according to claim 6, characterized in that, When the semantic reasoning model is a decoder unit, before inputting the estimated signal into the pre-trained semantic reasoning model for semantic reasoning, the method further includes: A training dataset is obtained, which includes transmission sample signals and data processing labels. The transmission sample signals are obtained by the base station based on the semantic sample signals corresponding to each task sender. The semantic sample signals are obtained based on the transmission sample signals. Based on the transmitted sample signal, an estimated sample signal is obtained, and the estimated sample signal is used to train the semantic reasoning model to obtain the data prediction result; The total loss value is calculated based on the data processing labels and the data prediction results; The semantic reasoning model is weighted based on the total loss value to obtain the trained semantic reasoning model.

10. A multi-task semantic communication system, characterized in that, include: Multiple task sending ends are provided, wherein each task sending end is used to obtain semantic feature signals according to any one of claims 1 to 5, and to send the semantic feature signals to the base station respectively, and each task sending end corresponds to a task receiving end; The base station is used to obtain the semantic feature vectors from multiple different task sending ends to obtain the transmission signal corresponding to each task receiving end, and to send the transmission signal to multiple task receiving ends one by one. Multiple task receiving ends are provided, each task receiving end being used to acquire transmission signals and, using the multi-task semantic communication method according to any one of claims 6 to 9, obtain target processing data corresponding to the communication task based on the transmission signals.

11. The multi-task semantic communication system according to claim 10, characterized in that, The step of the base station acquiring the semantic feature vectors from multiple different task sending ends to obtain the transmission signal corresponding to each task receiving end includes: Select target sending ends one by one from the task sending ends, and obtain the task receiving end corresponding to the target sending end as the target receiving end; The target transmission data is obtained based on the transmission parameters between the target transmitter and the base station and the semantic feature vector of the target transmitter; Interference transmission data is obtained based on the transmission parameters between the other task sending end and the base station and the semantic feature vector of the task sending end, and the interference transmission data is accumulated to obtain the total interference data; The transmission signal of the target receiver is obtained based on the target transmission data and the total interference data, and the transmission signal of each task receiver is obtained one by one.

12. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the multi-task semantic communication method according to any one of claims 1 to 9.

13. A storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the multitasking semantic communication method as described in any one of claims 1 to 9.

Images